2023-10-08T09:33:55.035 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T09:33:55.035 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} --owner scheduled_pdonnell@teuthology --name pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi --verbose --archive /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807 -- /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/orig.config.yaml 2023-10-08T09:33:55.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T09:33:55.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807 branch: wip-batrick-testing-20231007.154158 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7417807' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231007.154158 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de kclient: mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231007.154158 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8022 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de sleep_before_teardown: 0 subset: 128/256 suite: fs suite_branch: wip-batrick-testing-20231007.154158 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_20:01:29 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T09:33:55.182 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa; will attempt to use it 2023-10-08T09:33:55.183 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks 2023-10-08T09:33:55.183 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T09:33:55.185 INFO:teuthology.task.internal:Checking packages... 2023-10-08T09:33:55.208 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de' 2023-10-08T09:33:55.208 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T09:33:55.208 INFO:teuthology.packaging:ref: None 2023-10-08T09:33:55.208 INFO:teuthology.packaging:tag: None 2023-10-08T09:33:55.209 INFO:teuthology.packaging:branch: wip-batrick-testing-20231007.154158 2023-10-08T09:33:55.209 INFO:teuthology.packaging:sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:33:55.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231007.154158 2023-10-08T09:33:55.354 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6639.g7b3143ba 2023-10-08T09:33:55.354 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T09:33:55.360 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T09:33:55.361 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T09:33:55.500 INFO:teuthology.task.internal:Saving configuration 2023-10-08T09:33:55.523 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T09:33:55.530 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T09:33:55.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.795994', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:33:55.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.798030', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:33:55.591 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.799528', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:33:55.591 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T09:33:55.598 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-08T09:33:55.598 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-08T09:33:55.598 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-08T09:33:55.598 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T09:33:55.697 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd73e11cee0>, signals=[15]) 2023-10-08T09:33:55.698 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T09:33:55.704 INFO:teuthology.task.internal:Opening connections... 2023-10-08T09:33:55.704 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:33:55.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:33:55.774 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2023-10-08T09:33:55.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:33:55.846 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-10-08T09:33:55.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:33:55.917 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T09:33:55.924 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-10-08T09:33:55.945 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-10-08T09:33:55.945 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-10-08T09:33:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T09:33:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T09:33:56.006 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T09:33:56.007 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-10-08T09:33:56.026 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-10-08T09:33:56.046 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-10-08T09:33:56.046 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-10-08T09:33:56.109 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T09:33:56.109 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T09:33:56.109 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T09:33:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:33:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T09:33:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T09:33:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T09:33:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T09:33:56.111 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-10-08T09:33:56.134 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-10-08T09:33:56.156 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-10-08T09:33:56.156 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-10-08T09:33:56.216 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T09:33:56.217 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T09:33:56.218 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-10-08T09:33:56.234 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T09:33:56.245 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T09:33:56.322 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T09:33:56.322 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-10-08T09:33:56.325 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2023-10-08T09:33:56.327 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-10-08T09:33:56.346 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T09:33:56.352 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T09:33:56.353 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-10-08T09:33:56.384 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2023-10-08T09:33:56.389 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-10-08T09:33:56.415 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T09:33:56.477 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T09:33:56.491 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T09:33:56.491 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-10-08T09:33:56.491 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-10-08T09:33:56.492 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T09:33:56.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:33:56.511 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-10-08T09:33:56.571 INFO:teuthology.orchestra.run.smithi097.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-08T09:33:56.571 INFO:teuthology.task.kernel:Running kernel on smithi097: 4.18.0-372.9.1.el8.x86_64 2023-10-08T09:33:56.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2023-10-08T09:33:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:33:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-10-08T09:33:59.369 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-10-08T09:34:00.678 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-08T09:34:04.145 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 471 kB/s | 24 kB 00:00 2023-10-08T09:34:04.831 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-08T09:34:04.875 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-10-08T09:34:04.876 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2023-10-08T09:34:04.877 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-10-08T09:34:05.003 DEBUG:teuthology.orchestra.run.smithi097:> echo no | sudo yum reinstall kernel || true 2023-10-08T09:34:05.371 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:34:05.371 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-10-08T09:34:05.641 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:01 ago on Sun 08 Oct 2023 09:34:04 AM UTC. 2023-10-08T09:34:05.690 INFO:teuthology.orchestra.run.smithi097.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T09:34:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:Error: No packages marked for reinstall. 2023-10-08T09:34:05.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum reinstall -y kernel || true 2023-10-08T09:34:06.095 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:34:06.095 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-10-08T09:34:06.361 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:02 ago on Sun 08 Oct 2023 09:34:04 AM UTC. 2023-10-08T09:34:06.403 INFO:teuthology.orchestra.run.smithi097.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T09:34:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:Error: No packages marked for reinstall. 2023-10-08T09:34:06.475 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T09:34:06.555 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-08T09:34:06.555 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-08T09:34:06.556 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-08T09:34:06.556 DEBUG:teuthology.parallel:result is None 2023-10-08T09:34:06.556 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T09:34:06.563 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T09:34:06.564 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T09:34:06.566 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T09:34:06.568 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T09:34:06.589 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T09:34:06.609 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T09:34:06.649 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T09:34:06.649 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T09:34:06.651 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T09:34:06.654 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T09:34:06.694 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T09:34:06.701 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T09:34:06.701 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T09:34:06.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:34:06.739 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T09:34:06.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:34:06.758 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T09:34:06.760 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T09:34:06.781 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T09:34:06.801 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.811 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.822 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.834 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.838 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.852 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T09:34:06.853 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T09:34:06.861 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T09:34:06.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T09:34:06.864 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T09:34:06.882 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T09:34:06.925 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T09:34:06.934 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T09:34:06.934 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T09:34:06.937 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T09:34:06.954 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T09:34:06.985 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.087 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.213 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:34:07.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T09:34:07.280 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.325 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.378 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.508 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:34:07.508 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T09:34:07.577 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.624 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T09:34:07.678 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.766 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T09:34:07.811 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:34:07.811 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T09:34:07.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-10-08T09:34:07.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2023-10-08T09:34:07.886 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-10-08T09:34:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:34:07.951 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:34:07.987 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:34:08.286 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T09:34:08.294 INFO:teuthology.task.internal:Starting timer... 2023-10-08T09:34:08.294 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T09:34:08.382 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T09:34:08.441 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T09:34:08.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2023-10-08T09:34:08.499 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2023-10-08T09:34:08.501 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2023-10-08T09:34:08.556 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2023-10-08T09:34:08.558 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-10-08T09:34:08.609 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-10-08T09:34:08.611 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T09:34:08.611 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2023-10-08T09:34:08.647 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2023-10-08T09:34:08.647 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2023-10-08T09:34:08.666 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2023-10-08T09:34:08.666 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-10-08T09:34:08.691 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-10-08T09:34:08.691 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2023-10-08T09:34:08.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:34:08.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:34:08.719 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:34:08.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:34:08.747 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:34:08.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:34:08.774 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T09:34:08.775 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2023-10-08T09:34:08.802 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2023-10-08T09:34:08.831 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-10-08T09:34:08.857 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T09:34:08.868 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-08T09:34:08.874 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-08T09:34:09.014 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T09:34:09.029 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T09:34:09.030 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi178.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T09:43:16.758 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2023-10-08T09:43:16.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-10-08T09:43:16.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:43:16.829 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-10-08T09:43:16.896 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-10-08T09:43:16.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-08T09:43:16.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:43:16.965 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-10-08T09:43:17.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-08T09:43:17.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-08T09:43:17.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:43:17.103 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-10-08T09:43:17.131 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-08T09:43:17.132 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T09:43:17.143 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T09:43:17.143 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:43:17.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:43:17.146 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:43:17.146 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:43:17.149 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:43:17.149 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:43:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:43:17.180 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:43:17.192 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:43:17.196 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:43:17.206 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:43:17.222 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:43:17.272 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2023-10-08T09:43:17.279 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2023-10-08T09:43:17.284 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2023-10-08T09:43:17.291 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2023-10-08T09:43:17.292 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-10-08T09:43:17.300 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:43:17.304 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-10-08T09:43:17.305 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:43:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:43:17.318 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:43:17.320 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:43:17.334 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:43:17.381 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-10-08T09:43:17.384 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:43:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-10-08T09:43:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.398 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-10-08T09:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-10-08T09:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.405 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-08T09:43:17.408 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:43:17.409 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-08T09:43:17.409 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.409 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.409 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.409 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:43:17.410 INFO:teuthology.run_tasks:Running task pexec... 2023-10-08T09:43:17.419 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-08T09:43:17.419 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:43:17.419 DEBUG:teuthology.orchestra.run.smithi097:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:43:17.419 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi178.front.sepia.ceph.com 2023-10-08T09:43:17.420 DEBUG:teuthology.orchestra.run.smithi178:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:43:17.420 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi186.front.sepia.ceph.com 2023-10-08T09:43:17.420 DEBUG:teuthology.orchestra.run.smithi186:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:43:17.796 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:17.813 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:17.823 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:18.677 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:48 ago on Sun 08 Oct 2023 09:41:30 AM UTC. 2023-10-08T09:43:18.687 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:39 ago on Sun 08 Oct 2023 09:41:39 AM UTC. 2023-10-08T09:43:18.696 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:37 ago on Sun 08 Oct 2023 09:41:41 AM UTC. 2023-10-08T09:43:20.400 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout:Resetting modules: 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-08T09:43:20.401 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-08T09:43:20.402 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:20.406 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-08T09:43:20.407 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-08T09:43:20.407 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout:Resetting modules: 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-08T09:43:20.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:20.414 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout:Resetting modules: 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-10-08T09:43:20.416 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:21.356 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:39 ago on Sun 08 Oct 2023 09:41:41 AM UTC. 2023-10-08T09:43:21.372 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:41 ago on Sun 08 Oct 2023 09:41:39 AM UTC. 2023-10-08T09:43:21.372 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:50 ago on Sun 08 Oct 2023 09:41:30 AM UTC. 2023-10-08T09:43:24.074 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-10-08T09:43:24.518 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:24.524 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-08T09:43:24.542 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-08T09:43:24.992 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:25.001 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:25.334 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:54 ago on Sun 08 Oct 2023 09:41:30 AM UTC. 2023-10-08T09:43:25.853 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:46 ago on Sun 08 Oct 2023 09:41:39 AM UTC. 2023-10-08T09:43:25.866 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:44 ago on Sun 08 Oct 2023 09:41:41 AM UTC. 2023-10-08T09:43:27.185 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-10-08T09:43:27.315 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-10-08T09:43:27.315 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-10-08T09:43:27.316 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-10-08T09:43:27.316 INFO:teuthology.orchestra.run.smithi097.stdout:Installing group/module packages: 2023-10-08T09:43:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:43:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:43:27.316 INFO:teuthology.orchestra.run.smithi097.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:43:27.318 INFO:teuthology.orchestra.run.smithi097.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:43:27.319 INFO:teuthology.orchestra.run.smithi097.stdout:Installing module profiles: 2023-10-08T09:43:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools/common 2023-10-08T09:43:27.319 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:27.323 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-10-08T09:43:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools rhel8 2023-10-08T09:43:27.323 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:27.323 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-10-08T09:43:27.323 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-10-08T09:43:27.324 INFO:teuthology.orchestra.run.smithi097.stdout:Install 10 Packages 2023-10-08T09:43:27.324 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:27.324 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 18 M 2023-10-08T09:43:27.324 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 65 M 2023-10-08T09:43:27.324 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-10-08T09:43:27.350 INFO:teuthology.orchestra.run.smithi097.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 361 kB/s | 25 kB 00:00 2023-10-08T09:43:27.375 INFO:teuthology.orchestra.run.smithi097.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 433 kB/s | 41 kB 00:00 2023-10-08T09:43:27.400 INFO:teuthology.orchestra.run.smithi097.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 785 kB/s | 94 kB 00:00 2023-10-08T09:43:27.779 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-08T09:43:27.780 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-08T09:43:27.780 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-08T09:43:27.780 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-08T09:43:27.780 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-08T09:43:27.780 INFO:teuthology.orchestra.run.smithi186.stdout:Installing group/module packages: 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:43:27.781 INFO:teuthology.orchestra.run.smithi186.stdout:Installing module profiles: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools/common 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools rhel8 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Install 10 Packages 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 18 M 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 65 M 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-08T09:43:27.782 INFO:teuthology.orchestra.run.smithi186.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 663 kB/s | 41 kB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi097.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-10-08T09:43:27.783 INFO:teuthology.orchestra.run.smithi186.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 286 kB/s | 25 kB 00:00 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout:Installing group/module packages: 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:43:27.806 INFO:teuthology.orchestra.run.smithi178.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout:Installing module profiles: 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools/common 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling module streams: 2023-10-08T09:43:27.807 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools rhel8 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout:Install 10 Packages 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 18 M 2023-10-08T09:43:27.808 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 65 M 2023-10-08T09:43:27.809 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-10-08T09:43:27.809 INFO:teuthology.orchestra.run.smithi186.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 984 kB/s | 49 kB 00:00 2023-10-08T09:43:27.827 INFO:teuthology.orchestra.run.smithi097.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-10-08T09:43:27.827 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:43:27.827 INFO:teuthology.orchestra.run.smithi097.stdout:Total 33 MB/s | 18 MB 00:00 2023-10-08T09:43:27.834 INFO:teuthology.orchestra.run.smithi186.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 684 kB/s | 94 kB 00:00 2023-10-08T09:43:27.867 INFO:teuthology.orchestra.run.smithi186.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-08T09:43:27.879 INFO:teuthology.orchestra.run.smithi178.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 584 kB/s | 41 kB 00:00 2023-10-08T09:43:27.893 INFO:teuthology.orchestra.run.smithi186.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-08T09:43:27.903 INFO:teuthology.orchestra.run.smithi178.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 261 kB/s | 25 kB 00:00 2023-10-08T09:43:27.918 INFO:teuthology.orchestra.run.smithi186.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-08T09:43:27.929 INFO:teuthology.orchestra.run.smithi178.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 981 kB/s | 49 kB 00:00 2023-10-08T09:43:27.954 INFO:teuthology.orchestra.run.smithi178.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 644 kB/s | 94 kB 00:00 2023-10-08T09:43:27.988 INFO:teuthology.orchestra.run.smithi178.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-08T09:43:28.013 INFO:teuthology.orchestra.run.smithi178.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-08T09:43:28.018 INFO:teuthology.orchestra.run.smithi186.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-08T09:43:28.027 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-10-08T09:43:28.038 INFO:teuthology.orchestra.run.smithi178.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-10-08T09:43:28.046 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-10-08T09:43:28.046 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-10-08T09:43:28.130 INFO:teuthology.orchestra.run.smithi178.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-10-08T09:43:28.135 INFO:teuthology.orchestra.run.smithi186.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-10-08T09:43:28.226 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-10-08T09:43:28.227 INFO:teuthology.orchestra.run.smithi186.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-10-08T09:43:28.227 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:43:28.227 INFO:teuthology.orchestra.run.smithi186.stdout:Total 34 MB/s | 18 MB 00:00 2023-10-08T09:43:28.230 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-10-08T09:43:28.255 INFO:teuthology.orchestra.run.smithi178.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-10-08T09:43:28.356 INFO:teuthology.orchestra.run.smithi178.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-10-08T09:43:28.356 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:43:28.356 INFO:teuthology.orchestra.run.smithi178.stdout:Total 33 MB/s | 18 MB 00:00 2023-10-08T09:43:28.431 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-08T09:43:28.449 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-08T09:43:28.449 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-08T09:43:28.556 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-10-08T09:43:28.574 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-10-08T09:43:28.574 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-10-08T09:43:28.631 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-08T09:43:28.635 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-08T09:43:28.755 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-10-08T09:43:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-10-08T09:43:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-10-08T09:43:28.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:43:29.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:29.181 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-08T09:43:29.290 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-10-08T09:43:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:43:29.356 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:43:29.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:43:29.515 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:43:29.519 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:29.700 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:29.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:43:29.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:43:29.886 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:43:29.916 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:43:30.086 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:43:30.264 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:43:30.464 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:43:30.605 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:43:31.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:43:31.235 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:43:31.462 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:43:31.689 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:43:31.867 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:43:31.905 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:43:32.048 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:32.140 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:43:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:43:32.303 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:43:32.304 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:43:32.310 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:43:32.381 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:43:32.487 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:32.549 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:43:32.751 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:43:32.776 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:43:33.022 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:43:33.022 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:43:33.022 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:43:33.022 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:43:33.022 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:43:33.023 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:43:33.023 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:43:33.023 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:43:33.023 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:43:33.023 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:43:33.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:43:33.499 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:02:03 ago on Sun 08 Oct 2023 09:41:30 AM UTC. 2023-10-08T09:43:33.932 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:43:33.932 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:52 ago on Sun 08 Oct 2023 09:41:41 AM UTC. 2023-10-08T09:43:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:43:34.234 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:54 ago on Sun 08 Oct 2023 09:41:39 AM UTC. 2023-10-08T09:43:38.679 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-10-08T09:43:38.870 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:38.870 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-10-08T09:43:38.870 INFO:teuthology.orchestra.run.smithi097.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:38.870 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:38.870 INFO:teuthology.orchestra.run.smithi097.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:43:38.871 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-10-08T09:43:39.007 DEBUG:teuthology.parallel:result is None 2023-10-08T09:43:39.195 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-10-08T09:43:39.229 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-08T09:43:39.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:39.389 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-10-08T09:43:39.389 INFO:teuthology.orchestra.run.smithi178.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.389 INFO:teuthology.orchestra.run.smithi178.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:43:39.390 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:43:39.427 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-08T09:43:39.537 DEBUG:teuthology.parallel:result is None 2023-10-08T09:43:39.563 DEBUG:teuthology.parallel:result is None 2023-10-08T09:43:39.563 INFO:teuthology.run_tasks:Running task install... 2023-10-08T09:43:39.572 DEBUG:teuthology.task.install:project ceph 2023-10-08T09:43:39.572 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'}} 2023-10-08T09:43:39.572 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'} 2023-10-08T09:43:39.572 INFO:teuthology.task.install:Using flavor: default 2023-10-08T09:43:39.577 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T09:43:39.577 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T09:43:39.578 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:43:39.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:43:39.580 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:43:39.580 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:43:39.581 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:43:39.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:43:39.730 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:43:39.730 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:43:39.798 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:43:39.798 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:43:39.803 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:43:39.803 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:43:39.851 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:43:39.852 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:43:39.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:43:39.884 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:43:39.885 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:43:39.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:43:39.965 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:43:39.965 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:43:39.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:43:39.970 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:43:39.970 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:43:39.970 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:43:39.994 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:43:39.994 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:43:39.998 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:43:39.998 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:43:40.037 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:43:40.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:43:40.067 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:43:40.106 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2023-10-08T09:43:40.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2023-10-08T09:43:40.141 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:43:40.145 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:43:40.208 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2023-10-08T09:43:40.209 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2023-10-08T09:43:40.212 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-10-08T09:43:40.214 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-10-08T09:43:40.483 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:40.580 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:40.589 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:41.504 INFO:teuthology.orchestra.run.smithi097.stdout:44 files removed 2023-10-08T09:43:41.526 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-10-08T09:43:41.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:43:41.577 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:43:41.637 INFO:teuthology.orchestra.run.smithi178.stdout:44 files removed 2023-10-08T09:43:41.691 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:43:41.888 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:41.911 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:42.023 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-08T09:43:43.436 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 231 kB/s | 81 kB 00:00 2023-10-08T09:43:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 270 kB/s | 81 kB 00:00 2023-10-08T09:43:43.520 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 189 kB/s | 81 kB 00:00 2023-10-08T09:43:43.734 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-10-08T09:43:43.811 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-10-08T09:43:43.845 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-10-08T09:43:44.045 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-10-08T09:43:44.120 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-10-08T09:43:44.129 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-10-08T09:43:44.210 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-10-08T09:43:44.254 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-08T09:43:44.255 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-10-08T09:43:44.772 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 32 MB/s | 16 MB 00:00 2023-10-08T09:43:45.379 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-08T09:43:45.672 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-08T09:43:48.272 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 433 kB/s | 24 kB 00:00 2023-10-08T09:43:48.969 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-10-08T09:43:49.131 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 454 kB/s | 24 kB 00:00 2023-10-08T09:43:49.241 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-10-08T09:43:49.896 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-10-08T09:43:50.041 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-10-08T09:43:57.535 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-10-08T09:43:58.205 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-10-08T09:43:58.250 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-10-08T09:44:08.740 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:08.741 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:08.755 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:08.756 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:08.757 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:44:08.758 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:44:08.760 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:44:08.764 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:44:08.765 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:44:08.894 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:44:08.906 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:44:08.907 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:44:08.908 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:44:08.909 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:44:08.910 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:44:08.911 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:44:08.912 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:44:08.913 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:44:08.914 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:44:08.915 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:44:08.916 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:44:08.917 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:44:08.918 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 209 M 2023-10-08T09:44:08.919 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-08T09:44:09.361 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:09.362 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:09.376 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.377 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.378 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:44:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:44:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:44:09.383 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-10-08T09:44:09.384 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:44:09.474 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:09.475 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:44:09.489 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.490 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.491 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:44:09.493 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:44:09.495 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:44:09.498 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:44:09.499 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:44:09.516 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-08T09:44:09.528 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-08T09:44:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-10-08T09:44:09.528 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:44:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:44:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:44:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:44:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:44:09.533 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:44:09.534 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:44:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:44:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:44:09.537 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:44:09.538 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:44:09.539 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:44:09.540 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:44:09.540 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:44:09.540 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:44:09.540 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:44:09.540 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:44:09.541 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:44:09.542 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:44:09.542 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:44:09.542 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:44:09.542 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:44:09.542 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:44:09.543 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling module streams: 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql 10 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:44:09.544 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-08T09:44:09.545 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-08T09:44:09.545 INFO:teuthology.orchestra.run.smithi178.stdout:Install 128 Packages 2023-10-08T09:44:09.545 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 3 Packages 2023-10-08T09:44:09.545 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:44:09.546 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 209 M 2023-10-08T09:44:09.546 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-10-08T09:44:09.567 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 4.9 MB/s | 918 kB 00:00 2023-10-08T09:44:09.617 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 3.3 MB/s | 167 kB 00:00 2023-10-08T09:44:09.632 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-10-08T09:44:09.644 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-10-08T09:44:09.644 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-10-08T09:44:09.644 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-10-08T09:44:09.644 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2023-10-08T09:44:09.644 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:44:09.645 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:44:09.646 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:44:09.647 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:44:09.648 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:44:09.649 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:44:09.650 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:44:09.651 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:44:09.652 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:44:09.653 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:44:09.654 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:44:09.655 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:44:09.656 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:44:09.657 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql 10 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout:Install 128 Packages 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 3 Packages 2023-10-08T09:44:09.658 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:44:09.659 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 209 M 2023-10-08T09:44:09.659 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-10-08T09:44:09.843 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 10 MB/s | 2.3 MB 00:00 2023-10-08T09:44:09.851 INFO:teuthology.orchestra.run.smithi097.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-10-08T09:44:10.026 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 8.9 MB/s | 1.6 MB 00:00 2023-10-08T09:44:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-10-08T09:44:10.310 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 5.2 MB/s | 5.4 MB 00:01 2023-10-08T09:44:10.543 INFO:teuthology.orchestra.run.smithi097.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.3 MB/s | 918 kB 00:00 2023-10-08T09:44:10.560 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 8.7 MB/s | 4.6 MB 00:00 2023-10-08T09:44:10.685 INFO:teuthology.orchestra.run.smithi097.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 1.2 MB/s | 167 kB 00:00 2023-10-08T09:44:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.1 MB/s | 918 kB 00:00 2023-10-08T09:44:11.072 INFO:teuthology.orchestra.run.smithi178.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 1.0 MB/s | 167 kB 00:00 2023-10-08T09:44:12.110 INFO:teuthology.orchestra.run.smithi097.stdout:(4/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 1.6 MB/s | 2.3 MB 00:01 2023-10-08T09:44:12.322 INFO:teuthology.orchestra.run.smithi178.stdout:(4/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-10-08T09:44:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:(5/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 2.1 MB/s | 5.4 MB 00:02 2023-10-08T09:44:12.511 INFO:teuthology.orchestra.run.smithi097.stdout:(5/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-10-08T09:44:12.577 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el8 6.9 MB/s | 14 MB 00:02 2023-10-08T09:44:12.656 INFO:teuthology.orchestra.run.smithi178.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 4.9 MB/s | 1.6 MB 00:00 2023-10-08T09:44:12.660 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 292 kB/s | 24 kB 00:00 2023-10-08T09:44:12.686 INFO:teuthology.orchestra.run.smithi097.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 2.8 MB/s | 1.6 MB 00:00 2023-10-08T09:44:13.782 INFO:teuthology.orchestra.run.smithi178.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-10-08T09:44:14.411 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-common-18.0.0-6639.g7b3143ba.el8 4.4 MB/s | 23 MB 00:05 2023-10-08T09:44:14.636 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x8 4.1 MB/s | 18 MB 00:04 2023-10-08T09:44:14.661 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 1.0 MB/s | 261 kB 00:00 2023-10-08T09:44:14.745 INFO:teuthology.orchestra.run.smithi097.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 2.1 MB/s | 4.6 MB 00:02 2023-10-08T09:44:14.745 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): libcephfs-devel-18.0.0-6639.g7b3143ba 286 kB/s | 31 kB 00:00 2023-10-08T09:44:14.853 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 1.7 MB/s | 184 kB 00:00 2023-10-08T09:44:14.928 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 3.1 MB/s | 840 kB 00:00 2023-10-08T09:44:14.954 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): librados-devel-18.0.0-6639.g7b3143ba. 1.4 MB/s | 139 kB 00:00 2023-10-08T09:44:15.032 INFO:teuthology.orchestra.run.smithi178.stdout:(8/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x86 7.5 MB/s | 18 MB 00:02 2023-10-08T09:44:15.079 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 3.4 MB/s | 518 kB 00:00 2023-10-08T09:44:15.098 INFO:teuthology.orchestra.run.smithi178.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 367 kB/s | 24 kB 00:00 2023-10-08T09:44:15.188 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 413 kB/s | 45 kB 00:00 2023-10-08T09:44:15.305 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.0 MB/s | 125 kB 00:00 2023-10-08T09:44:15.422 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 1.6 MB/s | 197 kB 00:00 2023-10-08T09:44:15.589 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 2.2 MB/s | 371 kB 00:00 2023-10-08T09:44:15.772 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 1.9 MB/s | 365 kB 00:00 2023-10-08T09:44:15.872 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.0 MB/s | 107 kB 00:00 2023-10-08T09:44:15.981 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 870 kB/s | 94 kB 00:00 2023-10-08T09:44:16.840 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 3.7 MB/s | 6.9 MB 00:01 2023-10-08T09:44:16.956 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 1.5 MB/s | 179 kB 00:00 2023-10-08T09:44:17.032 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 3.4 MB/s | 3.6 MB 00:01 2023-10-08T09:44:17.065 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 212 kB/s | 23 kB 00:00 2023-10-08T09:44:17.124 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.4 MB/s | 132 kB 00:00 2023-10-08T09:44:17.624 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 3.2 MB/s | 1.8 MB 00:00 2023-10-08T09:44:17.741 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 2.0 MB/s | 242 kB 00:00 2023-10-08T09:44:17.875 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 355 kB/s | 47 kB 00:00 2023-10-08T09:44:17.975 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 145 kB/s | 15 kB 00:00 2023-10-08T09:44:18.092 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 2.2 MB/s | 261 kB 00:00 2023-10-08T09:44:18.193 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 1.9 MB/s | 190 kB 00:00 2023-10-08T09:44:18.301 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 167 kB/s | 18 kB 00:00 2023-10-08T09:44:18.576 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-10-08T09:44:18.643 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 882 kB/s | 59 kB 00:00 2023-10-08T09:44:18.719 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-10-08T09:44:18.827 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-10-08T09:44:18.894 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2023-10-08T09:44:18.970 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-10-08T09:44:19.037 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 495 kB/s | 33 kB 00:00 2023-10-08T09:44:19.103 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-10-08T09:44:19.245 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): ceph-mgr-diskprediction-local-18.0.0- 3.5 MB/s | 7.4 MB 00:02 2023-10-08T09:44:19.271 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-10-08T09:44:19.338 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-08T09:44:19.405 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-10-08T09:44:19.472 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 264 kB/s | 18 kB 00:00 2023-10-08T09:44:19.555 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-10-08T09:44:19.597 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-10-08T09:44:19.622 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 729 kB/s | 49 kB 00:00 2023-10-08T09:44:19.664 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 878 kB/s | 59 kB 00:00 2023-10-08T09:44:19.690 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 900 kB/s | 60 kB 00:00 2023-10-08T09:44:19.732 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-10-08T09:44:19.749 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-10-08T09:44:19.791 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 574 kB/s | 34 kB 00:00 2023-10-08T09:44:19.891 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-08T09:44:20.066 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 233 kB/s | 64 kB 00:00 2023-10-08T09:44:20.070 INFO:teuthology.orchestra.run.smithi097.stdout:(8/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x86 2.4 MB/s | 18 MB 00:07 2023-10-08T09:44:20.083 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 97 kB/s | 19 kB 00:00 2023-10-08T09:44:20.111 INFO:teuthology.orchestra.run.smithi097.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 589 kB/s | 24 kB 00:00 2023-10-08T09:44:20.125 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-10-08T09:44:20.142 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-10-08T09:44:20.184 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 472 kB/s | 27 kB 00:00 2023-10-08T09:44:20.209 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-10-08T09:44:20.251 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-10-08T09:44:20.277 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 915 kB/s | 61 kB 00:00 2023-10-08T09:44:20.319 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-10-08T09:44:20.344 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 344 kB/s | 23 kB 00:00 2023-10-08T09:44:20.369 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-08T09:44:20.382 INFO:teuthology.orchestra.run.smithi178.stdout:(10/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 9.3 MB/s | 49 MB 00:05 2023-10-08T09:44:20.461 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-10-08T09:44:20.532 INFO:teuthology.orchestra.run.smithi178.stdout:(11/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 2.1 MB/s | 14 MB 00:06 2023-10-08T09:44:20.557 INFO:teuthology.orchestra.run.smithi178.stdout:(12/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 1.5 MB/s | 261 kB 00:00 2023-10-08T09:44:20.583 INFO:teuthology.orchestra.run.smithi178.stdout:(13/131): libcephfs-devel-18.0.0-6639.g7b3143ba 621 kB/s | 31 kB 00:00 2023-10-08T09:44:20.691 INFO:teuthology.orchestra.run.smithi178.stdout:(14/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 6.1 MB/s | 840 kB 00:00 2023-10-08T09:44:20.717 INFO:teuthology.orchestra.run.smithi178.stdout:(15/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 1.4 MB/s | 184 kB 00:00 2023-10-08T09:44:20.736 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 37 MB/s | 14 MB 00:00 2023-10-08T09:44:20.750 INFO:teuthology.orchestra.run.smithi178.stdout:(16/131): librados-devel-18.0.0-6639.g7b3143ba. 2.3 MB/s | 139 kB 00:00 2023-10-08T09:44:20.762 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 367 kB/s | 110 kB 00:00 2023-10-08T09:44:20.853 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2023-10-08T09:44:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:(10/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 2.3 MB/s | 14 MB 00:06 2023-10-08T09:44:20.892 INFO:teuthology.orchestra.run.smithi178.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 2.9 MB/s | 518 kB 00:00 2023-10-08T09:44:20.996 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 209 kB/s | 54 kB 00:00 2023-10-08T09:44:20.996 INFO:teuthology.orchestra.run.smithi178.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 433 kB/s | 45 kB 00:00 2023-10-08T09:44:21.037 INFO:teuthology.orchestra.run.smithi097.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 1.5 MB/s | 261 kB 00:00 2023-10-08T09:44:21.071 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2023-10-08T09:44:21.076 INFO:teuthology.orchestra.run.smithi178.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.5 MB/s | 125 kB 00:00 2023-10-08T09:44:21.129 INFO:teuthology.orchestra.run.smithi097.stdout:(12/131): libcephfs-devel-18.0.0-6639.g7b3143ba 338 kB/s | 31 kB 00:00 2023-10-08T09:44:21.204 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 186 kB/s | 39 kB 00:00 2023-10-08T09:44:21.218 INFO:teuthology.orchestra.run.smithi178.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 1.4 MB/s | 197 kB 00:00 2023-10-08T09:44:21.418 INFO:teuthology.orchestra.run.smithi178.stdout:(21/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 10 MB/s | 6.9 MB 00:00 2023-10-08T09:44:21.443 INFO:teuthology.orchestra.run.smithi178.stdout:(22/131): python3-rados-18.0.0-6639.g7b3143ba.e 1.6 MB/s | 371 kB 00:00 2023-10-08T09:44:21.463 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.0 MB/s | 538 kB 00:00 2023-10-08T09:44:21.471 INFO:teuthology.orchestra.run.smithi097.stdout:(13/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 2.4 MB/s | 840 kB 00:00 2023-10-08T09:44:21.486 INFO:teuthology.orchestra.run.smithi178.stdout:(23/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 5.3 MB/s | 365 kB 00:00 2023-10-08T09:44:21.497 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 1.6 MB/s | 688 kB 00:00 2023-10-08T09:44:21.511 INFO:teuthology.orchestra.run.smithi178.stdout:(24/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.6 MB/s | 107 kB 00:00 2023-10-08T09:44:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:(25/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 2.2 MB/s | 94 kB 00:00 2023-10-08T09:44:21.546 INFO:teuthology.orchestra.run.smithi097.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 2.4 MB/s | 184 kB 00:00 2023-10-08T09:44:21.586 INFO:teuthology.orchestra.run.smithi178.stdout:(26/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 3.0 MB/s | 179 kB 00:00 2023-10-08T09:44:21.621 INFO:teuthology.orchestra.run.smithi097.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 1.8 MB/s | 139 kB 00:00 2023-10-08T09:44:21.645 INFO:teuthology.orchestra.run.smithi178.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6639.g 396 kB/s | 23 kB 00:00 2023-10-08T09:44:21.712 INFO:teuthology.orchestra.run.smithi178.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.9 MB/s | 132 kB 00:00 2023-10-08T09:44:21.822 INFO:teuthology.orchestra.run.smithi097.stdout:(16/131): libradosstriper1-18.0.0-6639.g7b3143b 2.5 MB/s | 518 kB 00:00 2023-10-08T09:44:21.838 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 418 kB/s | 157 kB 00:00 2023-10-08T09:44:21.971 INFO:teuthology.orchestra.run.smithi178.stdout:(29/131): ceph-common-18.0.0-6639.g7b3143ba.el8 1.9 MB/s | 23 MB 00:12 2023-10-08T09:44:22.021 INFO:teuthology.orchestra.run.smithi178.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 5.7 MB/s | 1.8 MB 00:00 2023-10-08T09:44:22.079 INFO:teuthology.orchestra.run.smithi178.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 4.1 MB/s | 242 kB 00:00 2023-10-08T09:44:22.130 INFO:teuthology.orchestra.run.smithi178.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 947 kB/s | 47 kB 00:00 2023-10-08T09:44:22.147 INFO:teuthology.orchestra.run.smithi097.stdout:(17/131): ceph-common-18.0.0-6639.g7b3143ba.el8 1.8 MB/s | 23 MB 00:12 2023-10-08T09:44:22.172 INFO:teuthology.orchestra.run.smithi178.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 350 kB/s | 15 kB 00:00 2023-10-08T09:44:22.206 INFO:teuthology.orchestra.run.smithi097.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 773 kB/s | 45 kB 00:00 2023-10-08T09:44:22.230 INFO:teuthology.orchestra.run.smithi178.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 4.4 MB/s | 261 kB 00:00 2023-10-08T09:44:22.281 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 5.1 MB/s | 49 MB 00:09 2023-10-08T09:44:22.289 INFO:teuthology.orchestra.run.smithi178.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 3.2 MB/s | 190 kB 00:00 2023-10-08T09:44:22.306 INFO:teuthology.orchestra.run.smithi097.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.2 MB/s | 125 kB 00:00 2023-10-08T09:44:22.306 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 232 kB/s | 187 kB 00:00 2023-10-08T09:44:22.349 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 1.4 MB/s | 713 kB 00:00 2023-10-08T09:44:22.373 INFO:teuthology.orchestra.run.smithi178.stdout:(36/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 4.2 MB/s | 3.6 MB 00:00 2023-10-08T09:44:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:(37/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 167 kB/s | 18 kB 00:00 2023-10-08T09:44:22.406 INFO:teuthology.orchestra.run.smithi097.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 1.9 MB/s | 197 kB 00:00 2023-10-08T09:44:22.506 INFO:teuthology.orchestra.run.smithi178.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 544 kB/s | 59 kB 00:00 2023-10-08T09:44:22.540 INFO:teuthology.orchestra.run.smithi097.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 2.7 MB/s | 371 kB 00:00 2023-10-08T09:44:22.557 INFO:teuthology.orchestra.run.smithi178.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-10-08T09:44:22.582 INFO:teuthology.orchestra.run.smithi178.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 997 kB/s | 75 kB 00:00 2023-10-08T09:44:22.614 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-10-08T09:44:22.632 INFO:teuthology.orchestra.run.smithi178.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2023-10-08T09:44:22.658 INFO:teuthology.orchestra.run.smithi178.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 580 kB/s | 44 kB 00:00 2023-10-08T09:44:22.681 INFO:teuthology.orchestra.run.smithi097.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 2.5 MB/s | 365 kB 00:00 2023-10-08T09:44:22.683 INFO:teuthology.orchestra.run.smithi178.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-10-08T09:44:22.708 INFO:teuthology.orchestra.run.smithi178.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 658 kB/s | 33 kB 00:00 2023-10-08T09:44:22.734 INFO:teuthology.orchestra.run.smithi178.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-10-08T09:44:22.757 INFO:teuthology.orchestra.run.smithi097.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.4 MB/s | 107 kB 00:00 2023-10-08T09:44:22.758 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): python3-jsonpatch-1.21-2.el8.noarch.r 60 kB/s | 27 kB 00:00 2023-10-08T09:44:22.784 INFO:teuthology.orchestra.run.smithi178.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-10-08T09:44:22.790 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 722 kB/s | 320 kB 00:00 2023-10-08T09:44:22.818 INFO:teuthology.orchestra.run.smithi178.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-10-08T09:44:22.843 INFO:teuthology.orchestra.run.smithi178.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-08T09:44:22.849 INFO:teuthology.orchestra.run.smithi097.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.0 MB/s | 94 kB 00:00 2023-10-08T09:44:22.868 INFO:teuthology.orchestra.run.smithi178.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-08T09:44:22.924 INFO:teuthology.orchestra.run.smithi178.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 219 kB/s | 18 kB 00:00 2023-10-08T09:44:22.969 INFO:teuthology.orchestra.run.smithi178.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-08T09:44:22.990 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.2 MB/s | 457 kB 00:00 2023-10-08T09:44:23.002 INFO:teuthology.orchestra.run.smithi178.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.7 MB/s | 1.2 MB 00:00 2023-10-08T09:44:23.024 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): python3-prettytable-0.7.2-14.el8.noar 107 kB/s | 44 kB 00:00 2023-10-08T09:44:23.028 INFO:teuthology.orchestra.run.smithi178.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-08T09:44:23.053 INFO:teuthology.orchestra.run.smithi178.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-08T09:44:23.078 INFO:teuthology.orchestra.run.smithi178.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-10-08T09:44:23.104 INFO:teuthology.orchestra.run.smithi178.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-10-08T09:44:23.108 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 768 kB/s | 64 kB 00:00 2023-10-08T09:44:23.129 INFO:teuthology.orchestra.run.smithi178.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2023-10-08T09:44:23.154 INFO:teuthology.orchestra.run.smithi178.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-08T09:44:23.179 INFO:teuthology.orchestra.run.smithi178.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-10-08T09:44:23.205 INFO:teuthology.orchestra.run.smithi178.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 370 kB/s | 19 kB 00:00 2023-10-08T09:44:23.230 INFO:teuthology.orchestra.run.smithi178.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 474 kB/s | 24 kB 00:00 2023-10-08T09:44:23.233 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.8 MB/s | 3.7 MB 00:00 2023-10-08T09:44:23.250 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 869 kB/s | 225 kB 00:00 2023-10-08T09:44:23.255 INFO:teuthology.orchestra.run.smithi178.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2023-10-08T09:44:23.281 INFO:teuthology.orchestra.run.smithi178.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2023-10-08T09:44:23.306 INFO:teuthology.orchestra.run.smithi178.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-10-08T09:44:23.331 INFO:teuthology.orchestra.run.smithi178.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-10-08T09:44:23.349 INFO:teuthology.orchestra.run.smithi097.stdout:(25/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 4.5 MB/s | 6.9 MB 00:01 2023-10-08T09:44:23.357 INFO:teuthology.orchestra.run.smithi178.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-08T09:44:23.373 INFO:teuthology.orchestra.run.smithi178.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-10-08T09:44:23.377 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 473 kB/s | 126 kB 00:00 2023-10-08T09:44:23.407 INFO:teuthology.orchestra.run.smithi178.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 460 kB/s | 23 kB 00:00 2023-10-08T09:44:23.416 INFO:teuthology.orchestra.run.smithi097.stdout:(26/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 2.6 MB/s | 179 kB 00:00 2023-10-08T09:44:23.432 INFO:teuthology.orchestra.run.smithi178.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-08T09:44:23.474 INFO:teuthology.orchestra.run.smithi178.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-10-08T09:44:23.475 INFO:teuthology.orchestra.run.smithi097.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6639.g 395 kB/s | 23 kB 00:00 2023-10-08T09:44:23.500 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 75 kB/s | 20 kB 00:00 2023-10-08T09:44:23.526 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 376 kB/s | 103 kB 00:00 2023-10-08T09:44:23.550 INFO:teuthology.orchestra.run.smithi097.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.7 MB/s | 132 kB 00:00 2023-10-08T09:44:23.568 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2023-10-08T09:44:23.767 INFO:teuthology.orchestra.run.smithi097.stdout:(29/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 3.9 MB/s | 3.6 MB 00:00 2023-10-08T09:44:23.826 INFO:teuthology.orchestra.run.smithi178.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 319 kB/s | 110 kB 00:00 2023-10-08T09:44:23.826 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-10-08T09:44:23.860 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 592 kB/s | 198 kB 00:00 2023-10-08T09:44:23.951 INFO:teuthology.orchestra.run.smithi097.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 4.4 MB/s | 1.8 MB 00:00 2023-10-08T09:44:23.952 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-10-08T09:44:23.992 INFO:teuthology.orchestra.run.smithi178.stdout:(72/131): ceph-mgr-diskprediction-local-18.0.0- 3.7 MB/s | 7.4 MB 00:02 2023-10-08T09:44:24.002 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-10-08T09:44:24.017 INFO:teuthology.orchestra.run.smithi178.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 284 kB/s | 54 kB 00:00 2023-10-08T09:44:24.059 INFO:teuthology.orchestra.run.smithi097.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 2.2 MB/s | 242 kB 00:00 2023-10-08T09:44:24.085 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-10-08T09:44:24.118 INFO:teuthology.orchestra.run.smithi097.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 816 kB/s | 47 kB 00:00 2023-10-08T09:44:24.153 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): postgresql-server-10.21-2.module+el8 25 MB/s | 5.1 MB 00:00 2023-10-08T09:44:24.177 INFO:teuthology.orchestra.run.smithi097.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 250 kB/s | 15 kB 00:00 2023-10-08T09:44:24.187 INFO:teuthology.orchestra.run.smithi178.stdout:(74/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-10-08T09:44:24.226 INFO:teuthology.orchestra.run.smithi178.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-10-08T09:44:24.251 INFO:teuthology.orchestra.run.smithi178.stdout:(76/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-10-08T09:44:24.269 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): postgresql-contrib-10.21-2.module+el 3.0 MB/s | 808 kB 00:00 2023-10-08T09:44:24.277 INFO:teuthology.orchestra.run.smithi097.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 2.6 MB/s | 261 kB 00:00 2023-10-08T09:44:24.311 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 295 kB/s | 47 kB 00:00 2023-10-08T09:44:24.352 INFO:teuthology.orchestra.run.smithi097.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 2.5 MB/s | 190 kB 00:00 2023-10-08T09:44:24.368 INFO:teuthology.orchestra.run.smithi178.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 215 kB/s | 39 kB 00:00 2023-10-08T09:44:24.395 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 4.9 MB/s | 1.5 MB 00:00 2023-10-08T09:44:24.411 INFO:teuthology.orchestra.run.smithi097.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 311 kB/s | 18 kB 00:00 2023-10-08T09:44:24.412 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 425 kB/s | 43 kB 00:00 2023-10-08T09:44:24.435 INFO:teuthology.orchestra.run.smithi178.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2023-10-08T09:44:24.437 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-10-08T09:44:24.477 INFO:teuthology.orchestra.run.smithi178.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.3 MB/s | 538 kB 00:00 2023-10-08T09:44:24.527 INFO:teuthology.orchestra.run.smithi178.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 988 kB/s | 157 kB 00:00 2023-10-08T09:44:24.537 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-10-08T09:44:24.553 INFO:teuthology.orchestra.run.smithi178.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-10-08T09:44:24.563 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 694 kB/s | 87 kB 00:00 2023-10-08T09:44:24.579 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 287 kB/s | 48 kB 00:00 2023-10-08T09:44:24.628 INFO:teuthology.orchestra.run.smithi178.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.6 MB/s | 713 kB 00:00 2023-10-08T09:44:24.670 INFO:teuthology.orchestra.run.smithi178.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 227 kB/s | 27 kB 00:00 2023-10-08T09:44:24.688 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 871 kB/s | 109 kB 00:00 2023-10-08T09:44:24.705 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-requests-oauthlib-1.0.0-1.el 255 kB/s | 43 kB 00:00 2023-10-08T09:44:24.739 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2023-10-08T09:44:24.787 INFO:teuthology.orchestra.run.smithi178.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-10-08T09:44:24.803 INFO:teuthology.orchestra.run.smithi097.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 782 kB/s | 306 kB 00:00 2023-10-08T09:44:24.814 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 172 kB/s | 19 kB 00:00 2023-10-08T09:44:24.840 INFO:teuthology.orchestra.run.smithi178.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.5 MB/s | 320 kB 00:00 2023-10-08T09:44:24.847 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-10-08T09:44:24.871 INFO:teuthology.orchestra.run.smithi178.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 219 kB/s | 44 kB 00:00 2023-10-08T09:44:24.872 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-10-08T09:44:24.886 INFO:teuthology.orchestra.run.smithi097.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 707 kB/s | 59 kB 00:00 2023-10-08T09:44:24.931 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-10-08T09:44:24.957 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 164 kB/s | 18 kB 00:00 2023-10-08T09:44:24.970 INFO:teuthology.orchestra.run.smithi097.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-10-08T09:44:24.971 INFO:teuthology.orchestra.run.smithi178.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 20 MB/s | 3.7 MB 00:00 2023-10-08T09:44:25.005 INFO:teuthology.orchestra.run.smithi178.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2023-10-08T09:44:25.015 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-10-08T09:44:25.038 INFO:teuthology.orchestra.run.smithi178.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.3 MB/s | 457 kB 00:00 2023-10-08T09:44:25.072 INFO:teuthology.orchestra.run.smithi178.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.8 MB/s | 126 kB 00:00 2023-10-08T09:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.6 MB/s | 605 kB 00:00 2023-10-08T09:44:25.103 INFO:teuthology.orchestra.run.smithi097.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-10-08T09:44:25.114 INFO:teuthology.orchestra.run.smithi178.stdout:(91/131): uuid-1.6.2-43.el8.x86_64.rpm 449 kB/s | 64 kB 00:00 2023-10-08T09:44:25.124 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 666 kB/s | 72 kB 00:00 2023-10-08T09:44:25.148 INFO:teuthology.orchestra.run.smithi178.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 184 kB/s | 20 kB 00:00 2023-10-08T09:44:25.179 INFO:teuthology.orchestra.run.smithi097.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 583 kB/s | 44 kB 00:00 2023-10-08T09:44:25.189 INFO:teuthology.orchestra.run.smithi178.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-10-08T09:44:25.200 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.6 MB/s | 2.3 MB 00:00 2023-10-08T09:44:25.215 INFO:teuthology.orchestra.run.smithi178.stdout:(94/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 726 kB/s | 103 kB 00:00 2023-10-08T09:44:25.225 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 597 kB/s | 60 kB 00:00 2023-10-08T09:44:25.267 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): fuse-2.9.7-15.el8.x86_64.rpm 494 kB/s | 83 kB 00:00 2023-10-08T09:44:25.298 INFO:teuthology.orchestra.run.smithi178.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2023-10-08T09:44:25.309 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 902 kB/s | 98 kB 00:00 2023-10-08T09:44:25.321 INFO:teuthology.orchestra.run.smithi097.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-10-08T09:44:25.367 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 405 kB/s | 58 kB 00:00 2023-10-08T09:44:25.399 INFO:teuthology.orchestra.run.smithi178.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-10-08T09:44:25.418 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 256 kB/s | 39 kB 00:00 2023-10-08T09:44:25.441 INFO:teuthology.orchestra.run.smithi178.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-10-08T09:44:25.454 INFO:teuthology.orchestra.run.smithi097.stdout:(43/131): ceph-mgr-diskprediction-local-18.0.0- 4.4 MB/s | 7.4 MB 00:01 2023-10-08T09:44:25.480 INFO:teuthology.orchestra.run.smithi097.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 208 kB/s | 33 kB 00:00 2023-10-08T09:44:25.524 INFO:teuthology.orchestra.run.smithi178.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-08T09:44:25.566 INFO:teuthology.orchestra.run.smithi178.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2023-10-08T09:44:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-08T09:44:25.666 INFO:teuthology.orchestra.run.smithi178.stdout:(100/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-10-08T09:44:25.688 INFO:teuthology.orchestra.run.smithi097.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-10-08T09:44:25.714 INFO:teuthology.orchestra.run.smithi097.stdout:(47/131): python3-certifi-2018.10.15-7.el8.noar 63 kB/s | 16 kB 00:00 2023-10-08T09:44:25.717 INFO:teuthology.orchestra.run.smithi178.stdout:(101/131): postgresql-contrib-10.21-2.module+el 4.1 MB/s | 808 kB 00:00 2023-10-08T09:44:25.718 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): NetworkManager-initscripts-updown-1. 459 kB/s | 138 kB 00:00 2023-10-08T09:44:25.742 INFO:teuthology.orchestra.run.smithi178.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 625 kB/s | 47 kB 00:00 2023-10-08T09:44:25.743 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 1.4 MB/s | 644 kB 00:00 2023-10-08T09:44:25.768 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 427 kB/s | 171 kB 00:00 2023-10-08T09:44:25.772 INFO:teuthology.orchestra.run.smithi097.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 973 kB/s | 81 kB 00:00 2023-10-08T09:44:25.784 INFO:teuthology.orchestra.run.smithi178.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 6.9 MB/s | 1.5 MB 00:00 2023-10-08T09:44:25.806 INFO:teuthology.orchestra.run.smithi097.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-10-08T09:44:25.809 INFO:teuthology.orchestra.run.smithi178.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2023-10-08T09:44:25.842 INFO:teuthology.orchestra.run.smithi178.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 424 kB/s | 43 kB 00:00 2023-10-08T09:44:25.848 INFO:teuthology.orchestra.run.smithi097.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-10-08T09:44:25.884 INFO:teuthology.orchestra.run.smithi178.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 638 kB/s | 48 kB 00:00 2023-10-08T09:44:25.885 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.7 MB/s | 798 kB 00:00 2023-10-08T09:44:25.910 INFO:teuthology.orchestra.run.smithi178.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-10-08T09:44:25.932 INFO:teuthology.orchestra.run.smithi097.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 584 kB/s | 49 kB 00:00 2023-10-08T09:44:25.977 INFO:teuthology.orchestra.run.smithi178.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2023-10-08T09:44:25.993 INFO:teuthology.orchestra.run.smithi178.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 393 kB/s | 43 kB 00:00 2023-10-08T09:44:26.019 INFO:teuthology.orchestra.run.smithi178.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.0 MB/s | 109 kB 00:00 2023-10-08T09:44:26.119 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 9.3 MB/s | 3.7 MB 00:00 2023-10-08T09:44:26.119 INFO:teuthology.orchestra.run.smithi178.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.4 MB/s | 560 kB 00:00 2023-10-08T09:44:26.144 INFO:teuthology.orchestra.run.smithi178.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2023-10-08T09:44:26.178 INFO:teuthology.orchestra.run.smithi178.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 118 kB/s | 19 kB 00:00 2023-10-08T09:44:26.278 INFO:teuthology.orchestra.run.smithi178.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 976 kB/s | 155 kB 00:00 2023-10-08T09:44:26.312 INFO:teuthology.orchestra.run.smithi178.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.3 MB/s | 559 kB 00:00 2023-10-08T09:44:26.329 INFO:teuthology.orchestra.run.smithi178.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 119 kB/s | 18 kB 00:00 2023-10-08T09:44:26.437 INFO:teuthology.orchestra.run.smithi178.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 22 MB/s | 2.3 MB 00:00 2023-10-08T09:44:26.449 INFO:teuthology.orchestra.run.smithi097.stdout:(52/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 7.7 MB/s | 49 MB 00:06 2023-10-08T09:44:26.471 INFO:teuthology.orchestra.run.smithi178.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2023-10-08T09:44:26.482 INFO:teuthology.orchestra.run.smithi097.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 107 kB/s | 59 kB 00:00 2023-10-08T09:44:26.496 INFO:teuthology.orchestra.run.smithi178.stdout:(119/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2023-10-08T09:44:26.510 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 4.8 MB/s | 3.7 MB 00:00 2023-10-08T09:44:26.514 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:44:26.514 INFO:teuthology.orchestra.run.smithi186.stdout:Total 12 MB/s | 209 MB 00:17 2023-10-08T09:44:26.547 INFO:teuthology.orchestra.run.smithi178.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 662 kB/s | 72 kB 00:00 2023-10-08T09:44:26.574 INFO:teuthology.orchestra.run.smithi097.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-10-08T09:44:26.614 INFO:teuthology.orchestra.run.smithi178.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 581 kB/s | 83 kB 00:00 2023-10-08T09:44:26.647 INFO:teuthology.orchestra.run.smithi178.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 398 kB/s | 60 kB 00:00 2023-10-08T09:44:26.666 INFO:teuthology.orchestra.run.smithi097.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 171 kB/s | 16 kB 00:00 2023-10-08T09:44:26.673 INFO:teuthology.orchestra.run.smithi178.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 782 kB/s | 98 kB 00:00 2023-10-08T09:44:26.733 INFO:teuthology.orchestra.run.smithi097.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 506 kB/s | 34 kB 00:00 2023-10-08T09:44:26.758 INFO:teuthology.orchestra.run.smithi097.stdout:(57/131): python3-natsort-7.1.1-2.el8.noarch.rp 195 kB/s | 60 kB 00:00 2023-10-08T09:44:26.773 INFO:teuthology.orchestra.run.smithi178.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 363 kB/s | 58 kB 00:00 2023-10-08T09:44:26.792 INFO:teuthology.orchestra.run.smithi097.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 1.2 MB/s | 1.2 MB 00:00 2023-10-08T09:44:26.818 INFO:teuthology.orchestra.run.smithi097.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-10-08T09:44:26.823 INFO:teuthology.orchestra.run.smithi178.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 220 kB/s | 39 kB 00:00 2023-10-08T09:44:26.834 INFO:teuthology.orchestra.run.smithi097.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 847 kB/s | 64 kB 00:00 2023-10-08T09:44:26.865 INFO:teuthology.orchestra.run.smithi178.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.3 MB/s | 644 kB 00:00 2023-10-08T09:44:26.868 INFO:teuthology.orchestra.run.smithi097.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 247 kB/s | 19 kB 00:00 2023-10-08T09:44:26.885 INFO:teuthology.orchestra.run.smithi097.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-10-08T09:44:26.890 INFO:teuthology.orchestra.run.smithi178.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.4 MB/s | 171 kB 00:00 2023-10-08T09:44:26.910 INFO:teuthology.orchestra.run.smithi097.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2023-10-08T09:44:26.941 INFO:teuthology.orchestra.run.smithi178.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.1 MB/s | 138 kB 00:00 2023-10-08T09:44:26.944 INFO:teuthology.orchestra.run.smithi097.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-10-08T09:44:26.977 INFO:teuthology.orchestra.run.smithi097.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-10-08T09:44:27.036 INFO:teuthology.orchestra.run.smithi097.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-10-08T09:44:27.061 INFO:teuthology.orchestra.run.smithi097.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2023-10-08T09:44:27.074 INFO:teuthology.orchestra.run.smithi178.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.9 MB/s | 798 kB 00:00 2023-10-08T09:44:27.078 INFO:teuthology.orchestra.run.smithi097.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 851 kB/s | 85 kB 00:00 2023-10-08T09:44:27.128 INFO:teuthology.orchestra.run.smithi097.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-10-08T09:44:27.154 INFO:teuthology.orchestra.run.smithi097.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 195 kB/s | 23 kB 00:00 2023-10-08T09:44:27.191 INFO:teuthology.orchestra.run.smithi178.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 12 MB/s | 3.7 MB 00:00 2023-10-08T09:44:27.321 INFO:teuthology.orchestra.run.smithi097.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 661 kB/s | 110 kB 00:00 2023-10-08T09:44:27.345 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-08T09:44:27.412 INFO:teuthology.orchestra.run.smithi097.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 4.9 MB/s | 1.7 MB 00:00 2023-10-08T09:44:27.446 INFO:teuthology.orchestra.run.smithi097.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2023-10-08T09:44:27.466 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-08T09:44:27.467 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-08T09:44:27.516 INFO:teuthology.orchestra.run.smithi178.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 5.9 MB/s | 3.7 MB 00:00 2023-10-08T09:44:27.519 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:44:27.519 INFO:teuthology.orchestra.run.smithi178.stdout:Total 12 MB/s | 209 MB 00:17 2023-10-08T09:44:27.697 INFO:teuthology.orchestra.run.smithi097.stdout:(74/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-10-08T09:44:27.721 INFO:teuthology.orchestra.run.smithi097.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2023-10-08T09:44:27.747 INFO:teuthology.orchestra.run.smithi097.stdout:(76/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 775 kB/s | 259 kB 00:00 2023-10-08T09:44:27.839 INFO:teuthology.orchestra.run.smithi097.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 273 kB/s | 39 kB 00:00 2023-10-08T09:44:27.872 INFO:teuthology.orchestra.run.smithi097.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-10-08T09:44:27.906 INFO:teuthology.orchestra.run.smithi097.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-10-08T09:44:27.957 INFO:teuthology.orchestra.run.smithi097.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-10-08T09:44:27.982 INFO:teuthology.orchestra.run.smithi097.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-10-08T09:44:28.032 INFO:teuthology.orchestra.run.smithi097.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 5.6 MB/s | 713 kB 00:00 2023-10-08T09:44:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 244 kB/s | 27 kB 00:00 2023-10-08T09:44:28.158 INFO:teuthology.orchestra.run.smithi097.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2023-10-08T09:44:28.241 INFO:teuthology.orchestra.run.smithi097.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 293 kB/s | 44 kB 00:00 2023-10-08T09:44:28.308 INFO:teuthology.orchestra.run.smithi097.stdout:(86/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-10-08T09:44:28.347 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-10-08T09:44:28.367 INFO:teuthology.orchestra.run.smithi097.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.6 MB/s | 457 kB 00:00 2023-10-08T09:44:28.425 INFO:teuthology.orchestra.run.smithi097.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-10-08T09:44:28.451 INFO:teuthology.orchestra.run.smithi097.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-10-08T09:44:28.466 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-10-08T09:44:28.466 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-10-08T09:44:28.467 INFO:teuthology.orchestra.run.smithi097.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 635 kB/s | 64 kB 00:00 2023-10-08T09:44:28.543 INFO:teuthology.orchestra.run.smithi097.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-10-08T09:44:28.568 INFO:teuthology.orchestra.run.smithi097.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-10-08T09:44:28.593 INFO:teuthology.orchestra.run.smithi097.stdout:(93/131): python3-jsonpointer-1.10-11.el8.noarc 140 kB/s | 20 kB 00:00 2023-10-08T09:44:28.594 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-08T09:44:28.599 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-08T09:44:28.627 INFO:teuthology.orchestra.run.smithi097.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-10-08T09:44:28.711 INFO:teuthology.orchestra.run.smithi097.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2023-10-08T09:44:28.802 INFO:teuthology.orchestra.run.smithi097.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 21 MB/s | 4.9 MB 00:00 2023-10-08T09:44:28.844 INFO:teuthology.orchestra.run.smithi097.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2023-10-08T09:44:28.936 INFO:teuthology.orchestra.run.smithi097.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-08T09:44:28.962 INFO:teuthology.orchestra.run.smithi097.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-10-08T09:44:29.070 INFO:teuthology.orchestra.run.smithi097.stdout:(100/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-10-08T09:44:29.112 INFO:teuthology.orchestra.run.smithi097.stdout:(101/131): postgresql-contrib-10.21-2.module+el 4.5 MB/s | 808 kB 00:00 2023-10-08T09:44:29.154 INFO:teuthology.orchestra.run.smithi097.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 7.8 MB/s | 1.5 MB 00:00 2023-10-08T09:44:29.234 INFO:teuthology.orchestra.run.smithi097.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 286 kB/s | 47 kB 00:00 2023-10-08T09:44:29.263 INFO:teuthology.orchestra.run.smithi097.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-10-08T09:44:29.321 INFO:teuthology.orchestra.run.smithi097.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-10-08T09:44:29.380 INFO:teuthology.orchestra.run.smithi097.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 411 kB/s | 48 kB 00:00 2023-10-08T09:44:29.405 INFO:teuthology.orchestra.run.smithi097.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-10-08T09:44:29.464 INFO:teuthology.orchestra.run.smithi097.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 612 kB/s | 87 kB 00:00 2023-10-08T09:44:29.489 INFO:teuthology.orchestra.run.smithi097.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 392 kB/s | 43 kB 00:00 2023-10-08T09:44:29.581 INFO:teuthology.orchestra.run.smithi097.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 621 kB/s | 109 kB 00:00 2023-10-08T09:44:29.619 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-10-08T09:44:29.623 INFO:teuthology.orchestra.run.smithi097.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-10-08T09:44:29.623 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-10-08T09:44:29.648 INFO:teuthology.orchestra.run.smithi097.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-10-08T09:44:29.715 INFO:teuthology.orchestra.run.smithi097.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 140 kB/s | 19 kB 00:00 2023-10-08T09:44:29.782 INFO:teuthology.orchestra.run.smithi097.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 4.1 MB/s | 559 kB 00:00 2023-10-08T09:44:29.807 INFO:teuthology.orchestra.run.smithi097.stdout:(115/131): python3-oauthlib-2.1.0-1.el8.noarch. 842 kB/s | 155 kB 00:00 2023-10-08T09:44:29.850 INFO:teuthology.orchestra.run.smithi097.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 136 kB/s | 18 kB 00:00 2023-10-08T09:44:29.929 INFO:teuthology.orchestra.run.smithi097.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-10-08T09:44:29.967 INFO:teuthology.orchestra.run.smithi097.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-10-08T09:44:30.042 INFO:teuthology.orchestra.run.smithi097.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-10-08T09:44:30.067 INFO:teuthology.orchestra.run.smithi097.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 523 kB/s | 72 kB 00:00 2023-10-08T09:44:30.101 INFO:teuthology.orchestra.run.smithi097.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 618 kB/s | 83 kB 00:00 2023-10-08T09:44:30.151 INFO:teuthology.orchestra.run.smithi097.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 549 kB/s | 60 kB 00:00 2023-10-08T09:44:30.185 INFO:teuthology.orchestra.run.smithi097.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 835 kB/s | 98 kB 00:00 2023-10-08T09:44:30.235 INFO:teuthology.orchestra.run.smithi097.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 431 kB/s | 58 kB 00:00 2023-10-08T09:44:30.269 INFO:teuthology.orchestra.run.smithi097.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 329 kB/s | 39 kB 00:00 2023-10-08T09:44:30.419 INFO:teuthology.orchestra.run.smithi097.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 2.7 MB/s | 644 kB 00:00 2023-10-08T09:44:30.672 INFO:teuthology.orchestra.run.smithi097.stdout:(127/131): NetworkManager-initscripts-updown-1. 341 kB/s | 138 kB 00:00 2023-10-08T09:44:30.703 INFO:teuthology.orchestra.run.smithi097.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 366 kB/s | 171 kB 00:00 2023-10-08T09:44:30.820 INFO:teuthology.orchestra.run.smithi097.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.7 MB/s | 798 kB 00:00 2023-10-08T09:44:30.953 INFO:teuthology.orchestra.run.smithi097.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 7.0 MB/s | 3.7 MB 00:00 2023-10-08T09:44:31.470 INFO:teuthology.orchestra.run.smithi097.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 4.7 MB/s | 3.7 MB 00:00 2023-10-08T09:44:31.473 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:44:31.473 INFO:teuthology.orchestra.run.smithi097.stdout:Total 9.6 MB/s | 209 MB 00:21 2023-10-08T09:44:32.283 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-10-08T09:44:32.403 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-10-08T09:44:32.403 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-10-08T09:44:33.481 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-08T09:44:33.522 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-10-08T09:44:33.527 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-10-08T09:44:33.662 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:44:33.832 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:44:34.038 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-08T09:44:34.222 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:34.337 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:34.486 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-10-08T09:44:34.614 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:34.674 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:44:34.955 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:44:35.024 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:35.231 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:35.300 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-08T09:44:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:35.609 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:35.626 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:35.814 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:35.876 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:35.932 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:36.097 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:36.191 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:36.273 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:36.475 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:36.531 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:44:36.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:44:36.880 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:37.010 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:44:37.092 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:37.221 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:37.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:37.414 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:44:37.483 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:37.621 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:44:37.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:44:37.755 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:37.858 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:37.905 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:44:38.005 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:38.069 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:38.103 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:44:38.238 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:38.374 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:38.474 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:44:38.522 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:38.537 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-10-08T09:44:38.706 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:44:38.724 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:44:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:38.891 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:44:38.996 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:39.098 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-08T09:44:39.120 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:39.158 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:39.232 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:39.272 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:39.300 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:39.375 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:39.413 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:44:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:39.692 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:39.879 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:40.067 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:44:40.072 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:40.308 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:40.475 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:40.668 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:44:40.782 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:40.968 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:44:41.053 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-08T09:44:41.098 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:41.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:44:41.266 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:41.274 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-08T09:44:41.438 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:44:41.492 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-08T09:44:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:44:41.686 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-08T09:44:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:44:41.894 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-08T09:44:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:42.140 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:44:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:44:42.281 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-08T09:44:42.344 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:44:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:44:42.483 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:44:42.494 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-08T09:44:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:42.712 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-08T09:44:42.735 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:44:42.891 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:42.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-08T09:44:42.955 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:44:43.091 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-08T09:44:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:44:43.360 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:44:43.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:44:43.416 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:43.564 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:44:43.702 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:44:43.720 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-08T09:44:43.966 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-08T09:44:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:44:44.216 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:44.221 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-08T09:44:44.404 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:44.600 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:44:44.812 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:44:44.932 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-08T09:44:44.982 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:45.211 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-08T09:44:45.496 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-08T09:44:45.682 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:45.755 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:44:46.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-08T09:44:46.166 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:46.243 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-08T09:44:46.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-08T09:44:46.489 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-08T09:44:46.655 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-08T09:44:46.682 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:44:46.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-08T09:44:46.872 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:44:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:44:47.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:44:47.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:44:47.435 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:44:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:47.575 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:44:47.745 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-08T09:44:47.792 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-08T09:44:47.897 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-08T09:44:48.036 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:44:48.073 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-08T09:44:48.225 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-08T09:44:48.242 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:44:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:44:48.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:44:48.446 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-08T09:44:48.628 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:44:48.636 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-08T09:44:48.714 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-08T09:44:48.798 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-08T09:44:48.967 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-08T09:44:48.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-08T09:44:49.001 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-08T09:44:49.209 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-08T09:44:49.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-08T09:44:49.265 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-08T09:44:49.360 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-08T09:44:49.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:49.456 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-08T09:44:49.557 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-08T09:44:49.667 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-08T09:44:49.748 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-08T09:44:49.925 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-08T09:44:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-08T09:44:50.220 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-08T09:44:50.453 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-08T09:44:50.468 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-08T09:44:50.671 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-08T09:44:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-08T09:44:50.875 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-08T09:44:50.902 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-08T09:44:51.041 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-08T09:44:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-08T09:44:51.187 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:44:51.215 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-08T09:44:51.465 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-08T09:44:51.486 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-08T09:44:51.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-08T09:44:51.655 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-08T09:44:51.736 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:44:51.759 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-08T09:44:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-08T09:44:52.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-08T09:44:52.068 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-08T09:44:52.243 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-08T09:44:52.243 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-08T09:44:52.269 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:44:52.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-08T09:44:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-08T09:44:52.464 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:44:52.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-08T09:44:52.687 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:44:52.720 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-08T09:44:52.745 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:52.816 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:52.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-08T09:44:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-08T09:44:53.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-08T09:44:53.094 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-08T09:44:53.219 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-08T09:44:53.259 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-08T09:44:53.290 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-08T09:44:53.402 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-08T09:44:53.428 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:53.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-08T09:44:53.511 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-08T09:44:53.635 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-08T09:44:53.724 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-08T09:44:53.774 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-08T09:44:53.830 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-08T09:44:53.887 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-08T09:44:53.931 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:53.998 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:54.035 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:44:54.198 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:44:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-08T09:44:54.227 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-08T09:44:54.450 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-08T09:44:54.496 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:44:54.518 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-08T09:44:54.614 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:54.693 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:54.724 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-08T09:44:54.726 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:44:54.865 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:44:54.901 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-08T09:44:54.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-08T09:44:55.005 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:44:55.081 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-08T09:44:55.185 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:44:55.213 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:44:55.236 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-08T09:44:55.355 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:44:55.383 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:44:55.521 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:44:55.580 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-08T09:44:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:44:55.697 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:44:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-08T09:44:55.867 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:44:55.953 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-08T09:44:56.077 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:44:56.177 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-08T09:44:56.222 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:44:56.315 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:44:56.345 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-08T09:44:56.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:44:56.483 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:44:56.501 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-08T09:44:56.542 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:44:56.699 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:44:56.733 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:44:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-08T09:44:56.901 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:44:56.961 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:44:57.058 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:44:57.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-08T09:44:57.159 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:44:57.302 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-08T09:44:57.308 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:44:57.316 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:44:57.445 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:44:57.538 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-08T09:44:57.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-08T09:44:57.701 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:44:57.832 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-08T09:44:57.917 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:44:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:44:58.111 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:44:58.355 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-08T09:44:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:44:58.386 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:44:58.551 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-08T09:44:58.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:44:58.681 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:44:58.689 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:58.745 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:44:58.772 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:44:58.927 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:44:58.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:44:59.009 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-08T09:44:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-08T09:44:59.220 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:44:59.339 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:44:59.485 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:44:59.517 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:44:59.671 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:44:59.780 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:45:00.004 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:45:00.102 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:45:00.160 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:45:00.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:45:00.367 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:45:00.453 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:45:00.521 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:45:00.604 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:45:00.619 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:00.705 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:00.716 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:45:00.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:45:00.873 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:45:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:45:01.067 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:45:01.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:45:01.221 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:45:01.455 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:45:01.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:45:01.666 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:45:01.910 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:45:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:45:02.294 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:45:02.362 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:45:02.454 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:02.517 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:02.552 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:45:02.713 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:45:02.784 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:45:03.874 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:45:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:45:04.422 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:45:04.547 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:04.756 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:45:04.768 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:05.035 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:45:05.250 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:45:05.363 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:05.622 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:05.692 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:05.860 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:06.068 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:45:06.230 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:06.304 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:45:06.380 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:10.174 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:10.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:11.009 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:45:11.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:11.353 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:45:11.479 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:24.392 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-08T09:45:24.393 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:45:24.749 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:26.338 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:26.519 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:26.739 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:26.924 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:27.161 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /run 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-10-08T09:45:27.162 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:45:27.386 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:27.483 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:27.540 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:27.557 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:29.074 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:29.216 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:29.306 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:29.436 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:29.471 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:29.570 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:45:29.625 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:29.747 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:45:29.923 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:30.074 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /run 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-10-08T09:45:30.075 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:45:30.089 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:30.144 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:30.274 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:30.419 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:30.435 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:31.395 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:31.597 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:31.689 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:31.954 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:45:32.052 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:45:32.165 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:32.176 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:32.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:32.293 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:32.296 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:45:32.385 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:45:32.456 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:32.467 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:32.499 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:45:32.552 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:32.910 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:33.011 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:45:33.168 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:33.185 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:45:33.206 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:33.791 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:45:33.969 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:34.003 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:34.131 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:34.165 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:34.176 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:34.331 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:34.340 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:34.431 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:34.701 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:45:34.839 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:35.056 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:35.092 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:35.109 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:45:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:45:35.265 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:35.276 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:35.442 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:45:35.659 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:35.880 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:45:36.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:36.609 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:45:36.819 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:36.853 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:37.006 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:37.048 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:37.105 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:45:37.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:37.284 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:37.375 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:45:37.578 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:37.708 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:45:37.731 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:37.918 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:37.952 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:45:38.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:38.104 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:45:39.410 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:45:39.481 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:45:39.787 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:45:39.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:39.884 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:45:39.946 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:40.034 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:40.137 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:40.393 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:45:40.576 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:41.025 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-08T09:45:41.316 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-08T09:45:41.474 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 119/134 2023-10-08T09:45:41.650 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:41.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:45:41.980 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:42.401 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:42.677 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:45:42.830 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:43.022 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:43.891 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-08T09:45:44.191 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-08T09:45:44.220 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 119/134 2023-10-08T09:45:44.463 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:44.481 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:44.584 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:44.613 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:45:44.759 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:45:44.887 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:45:44.947 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:45.180 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:45:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:45:45.456 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:45:45.557 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:45:45.604 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:45.612 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:45.708 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:45.798 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-08T09:45:45.906 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:45:45.970 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:46.442 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:46.443 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:46.443 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:46.443 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:46.457 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:46.457 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:46.677 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:46.818 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-08T09:45:47.951 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:47.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:47.954 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:47.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:45:47.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:45:47.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-08T09:45:47.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 119/134 2023-10-08T09:45:47.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:47.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:45:47.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:47.965 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:47.965 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:47.965 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:45:47.966 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:45:47.967 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:45:47.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:45:47.969 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:45:47.970 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:45:47.971 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-08T09:45:47.972 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-08T09:45:47.973 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-08T09:45:47.974 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-08T09:45:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-08T09:45:47.976 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:47.977 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:45:48.258 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:45:48.572 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:45:48.754 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:48.762 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:48.932 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-08T09:45:49.103 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:49.182 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:49.404 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:49.416 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:49.416 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:49.615 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:49.615 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:49.837 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:49.920 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:49.920 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:52 ago on Sun 08 Oct 2023 09:43:57 AM UTC. 2023-10-08T09:45:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:50.133 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:50.424 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:50.442 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:45:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:45:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:45:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:45:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:45:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:45:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-08T09:45:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-08T09:45:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-08T09:45:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-08T09:45:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-08T09:45:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:45:50.619 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:45:50.882 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:45:51.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:45:51.451 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:45:51.590 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:51.599 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:51.771 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-08T09:45:51.973 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:52.057 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:45:52.282 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:52.295 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:45:52.295 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:52.510 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:52.510 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:52.541 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.753 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.754 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.755 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.756 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:45:52.757 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:45:52.758 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.759 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:45:52.760 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:52.761 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:45:52.762 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:45:52.763 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:45:52.764 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:45:52.765 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:45:52.766 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:45:52.767 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:45:52.768 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:45:52.768 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:45:52.768 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:45:52.768 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-08T09:45:52.941 DEBUG:teuthology.parallel:result is None 2023-10-08T09:45:52.981 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:53.107 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:53.107 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:56 ago on Sun 08 Oct 2023 09:43:57 AM UTC. 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:45:53.494 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:45:53.495 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:45:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:45:53.497 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:45:53.498 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:45:53.499 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-08T09:45:53.500 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-08T09:45:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-08T09:45:53.502 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-08T09:45:53.502 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-08T09:45:53.502 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-08T09:45:53.503 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-08T09:45:53.504 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-08T09:45:53.505 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:45:53.506 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:45:53.507 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:45:55.875 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-10-08T09:45:55.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:45:55.955 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:58 ago on Sun 08 Oct 2023 09:43:57 AM UTC. 2023-10-08T09:45:56.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:45:56.089 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.090 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.091 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:45:56.092 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.093 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:45:56.094 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:45:56.095 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:45:56.096 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:45:56.097 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.098 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:45:56.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:45:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:45:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-08T09:45:56.101 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-08T09:45:56.267 DEBUG:teuthology.parallel:result is None 2023-10-08T09:46:01.153 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-10-08T09:46:01.366 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:46:01.366 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.367 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.369 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.370 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.371 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:46:01.372 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:46:01.373 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:46:01.374 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:46:01.375 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:46:01.375 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:46:01.375 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:46:01.375 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.375 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:46:01.376 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:46:01.377 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:46:01.378 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:46:01.378 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:46:01.378 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:46:01.378 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:46:01.378 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:46:01.379 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:46:01.380 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:46:01.381 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:46:01.382 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:46:01.383 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:46:01.384 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:46:01.384 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:46:01.384 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-10-08T09:46:01.552 DEBUG:teuthology.parallel:result is None 2023-10-08T09:46:01.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:01.776 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:46:01.799 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:01.799 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:01.800 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:46:01.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:02.010 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:46:02.037 INFO:teuthology.orchestra.run.smithi178.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:02.038 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:02.038 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:46:02.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:02.256 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:46:02.284 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:02.285 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:46:02.285 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:46:02.285 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T09:46:02.286 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:46:02.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:46:02.329 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:46:02.329 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:46:02.376 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:46:02.377 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:46:02.423 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T09:46:02.423 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:46:02.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:46:02.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:46:02.519 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:46:02.519 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:46:02.551 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:46:02.622 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:46:02.622 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:46:02.653 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:46:02.723 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T09:46:02.724 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:46:02.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:46:02.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:46:02.817 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:46:02.818 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:46:02.850 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:46:02.987 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:46:02.988 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:46:03.018 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:46:03.087 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T09:46:03.087 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:46:03.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:46:03.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:46:03.180 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:46:03.180 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:46:03.212 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:46:03.282 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:46:03.282 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:46:03.313 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:46:03.383 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-08T09:46:03.475 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'} 2023-10-08T09:46:03.476 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:03.476 INFO:tasks.cephadm:Cluster fsid is 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:03.477 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-08T09:46:03.477 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.b': '172.21.15.178', 'mon.c': '172.21.15.186'} 2023-10-08T09:46:03.477 INFO:tasks.cephadm:First mon is mon.a on smithi097 2023-10-08T09:46:03.477 INFO:tasks.cephadm:First mgr is x 2023-10-08T09:46:03.477 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-08T09:46:03.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2023-10-08T09:46:03.505 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2023-10-08T09:46:03.538 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2023-10-08T09:46:03.567 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-08T09:46:03.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:03.814 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73096/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6639-g7b3143ba', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6639.g7b3143ba'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-07 17:16:01.199993', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231007.154158', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-08T09:46:04.031 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20231007.154158, sha1 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:46:04.031 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm 2023-10-08T09:46:04.031 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:04.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:46 /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:04.620 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.014 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:46 /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.014 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.599 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:46 /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.599 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.620 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.642 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:46:05.677 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de on all hosts... 2023-10-08T09:46:05.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:46:05.680 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:46:05.684 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:46:10.202 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:46:10.620 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:46:11.002 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:46:39.594 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-08T09:46:39.595 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T09:46:40.195 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:46:41.419 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-08T09:46:41.419 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:46:41.420 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:46:41.420 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2023-10-08T09:46:41.420 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:46:41.420 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2023-10-08T09:46:41.420 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-08T09:46:41.440 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2023-10-08T09:46:41.478 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2023-10-08T09:46:41.515 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2023-10-08T09:46:41.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2023-10-08T09:46:41.574 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2023-10-08T09:46:41.604 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2023-10-08T09:46:41.631 INFO:tasks.cephadm:Writing seed config... 2023-10-08T09:46:41.633 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-08T09:46:41.633 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-08T09:46:41.633 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-08T09:46:41.634 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-08T09:46:41.635 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-08T09:46:41.636 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:46:41.636 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-08T09:46:41.657 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7ec07914-65bf-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-10-08T09:46:41.657 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-08T09:46:41.701 DEBUG:teuthology.orchestra.run.smithi097:mgr.x> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-08T09:46:41.767 INFO:journalctl@ceph.mon.a.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:46:41.811 INFO:tasks.cephadm:Bootstrapping... 2023-10-08T09:46:41.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de -v bootstrap --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:46:41.839 INFO:journalctl@ceph.mgr.x.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:46:41.982 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:46:41.982 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', '-v', 'bootstrap', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2023-10-08T09:46:42.133 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:46:42.133 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-08T09:46:42.133 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2023-10-08T09:46:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:46:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2023-10-08T09:46:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2023-10-08T09:46:42.282 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T09:46:42.282 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T09:46:42.288 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T09:46:42.288 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:46:42.294 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2023-10-08T09:46:42.299 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2023-10-08T09:46:42.299 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2023-10-08T09:46:42.299 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2023-10-08T09:46:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:46:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-08T09:46:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2023-10-08T09:46:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2023-10-08T09:46:42.447 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T09:46:42.447 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T09:46:42.453 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T09:46:42.453 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:46:42.458 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2023-10-08T09:46:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2023-10-08T09:46:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2023-10-08T09:46:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2023-10-08T09:46:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:42.464 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 139706151045608 on /run/cephadm/7ec07914-65bf-11ee-8db5-212e2dc638e7.lock 2023-10-08T09:46:42.465 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 139706151045608 acquired on /run/cephadm/7ec07914-65bf-11ee-8db5-212e2dc638e7.lock 2023-10-08T09:46:42.465 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2023-10-08T09:46:42.465 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2023-10-08T09:46:42.465 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2023-10-08T09:46:42.469 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-08T09:46:42.469 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2023-10-08T09:46:42.472 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-08T09:46:42.472 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-10-08T09:46:42.472 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::7efb:78c9:efd1:4682/64 scope link noprefixroute 2023-10-08T09:46:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T09:46:42.476 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2023-10-08T09:46:42.476 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2023-10-08T09:46:42.476 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-08T09:46:42.476 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-08T09:46:42.477 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:1016a809630cb60ade9603521911a0725a35bac1f67b6518836ae2dd33bb1dad 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-08T09:46:42.921 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Storing signatures 2023-10-08T09:46:43.668 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T09:46:43.668 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T09:46:43.668 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2023-10-08T09:46:44.420 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2023-10-08T09:46:44.420 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2023-10-08T09:46:45.168 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCEeiJlWFJiLRAAnoKw3LUnJBcK0Uxmk5cNhg== 2023-10-08T09:46:45.923 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCFeiJlCZCzHhAArjk8hwAC7tA+YqDqSPpmsw== 2023-10-08T09:46:46.671 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQCGeiJl/r9JDxAA7UEFsIpBkGhH7KwYGVr/WA== 2023-10-08T09:46:46.671 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2023-10-08T09:46:47.319 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = pacific 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T09:46:47.320 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:46:47.321 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2023-10-08T09:46:47.964 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2023-10-08T09:46:48.082 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-08T09:46:48.167 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-08T09:46:48.276 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target. 2023-10-08T09:46:48.276 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target. 2023-10-08T09:46:48.451 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T09:46:48.451 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-08T09:46:48.536 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target.wants/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@.service. 2023-10-08T09:46:48.838 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:48 smithi097 systemd[1]: Starting Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:46:49.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:48 smithi097 podman[98948]: 2023-10-08T09:46:49.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:49 smithi097 bash[98948]: a2d1af7e9b5d66aa3d2d33cf4a6d9649bb772033655f2685f48dff292f9b0fb0 2023-10-08T09:46:49.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:49 smithi097 ceph-mon[98988]: mkfs 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:49.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:49 smithi097 ceph-mon[98988]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:46:49.140 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:46:49.140 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2023-10-08T09:46:49.147 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:46:49.147 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:46:49.147 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2023-10-08T09:46:49.147 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T09:46:49.147 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2023-10-08T09:46:49.148 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2023-10-08T09:46:49.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:49 smithi097 systemd[1]: Started Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:46:50.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.629363s) 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-08T09:46:50.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2023-10-08T09:46:50.097 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: monmap e1: 1 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: fsmap 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: mgrmap e1: no daemons active 2023-10-08T09:46:50.291 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:50 smithi097 ceph-mon[98988]: from='client.? 172.21.15.97:0/1004556946' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-08T09:46:50.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.937 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:50.938 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T09:46:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2023-10-08T09:46:51.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:51 smithi097 ceph-mon[98988]: from='client.? 172.21.15.97:0/3624411399' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T09:46:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:51 smithi097 ceph-mon[98988]: from='client.? 172.21.15.97:0/3624411399' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-08T09:46:51.778 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2023-10-08T09:46:52.072 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:51 smithi097 systemd[1]: Stopping Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:46:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:52 smithi097 ceph-mon[98988]: from='client.? 172.21.15.97:0/1299463309' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:46:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:52 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a[98964]: 2023-10-08T09:46:52.119+0000 7f905cb9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:46:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:52 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a[98964]: 2023-10-08T09:46:52.119+0000 7f905cb9e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-08T09:46:53.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:53 smithi097 bash[99322]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T09:46:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:54 smithi097 systemd[1]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-08T09:46:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:54 smithi097 systemd[1]: Stopped Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:46:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:54 smithi097 systemd[1]: Starting Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:54 smithi097 podman[99451]: 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 7 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: pidfile_write: ignore empty --pid-file 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 bash[99451]: b752fcf3b8f7198afaa1355d447c34acb73f402dbd1ace928b438f2052dd83cb 2023-10-08T09:46:55.103 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: load: jerasure load: lrc 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: RocksDB version: 7.9.2 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Git sha 0 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: DB SUMMARY 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: DB Session ID: H9DKPC4UT4XYAHIL3YNS 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: CURRENT file: CURRENT 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89108 ; 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.error_if_exists: 0 2023-10-08T09:46:55.104 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.create_if_missing: 0 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.paranoid_checks: 1 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.env: 0x55ba0c22ada0 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.info_log: 0x55ba0e533640 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.statistics: (nil) 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.use_fsync: 0 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_log_file_size: 0 2023-10-08T09:46:55.105 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_fallocate: 1 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.use_direct_reads: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.db_log_dir: 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.wal_dir: 2023-10-08T09:46:55.106 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.write_buffer_manager: 0x55ba0e4eb360 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T09:46:55.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.unordered_write: 0 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.row_cache: None 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.wal_filter: None 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T09:46:55.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.two_write_queues: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.wal_compression: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.atomic_flush: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.log_readahead_size: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T09:46:55.109 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_background_jobs: 2 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_background_compactions: -1 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_subcompactions: 1 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T09:46:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_open_files: -1 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_background_flushes: -1 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Compression algorithms supported: 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kZSTD supported: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kXpressCompression supported: 0 2023-10-08T09:46:55.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kZlibCompression supported: 1 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kSnappyCompression supported: 1 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kLZ4Compression supported: 1 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: kBZip2Compression supported: 0 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.merge_operator: 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_filter: None 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T09:46:55.113 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba0e533740) 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2023-10-08T09:46:55.114 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x55ba0e5891f0 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2023-10-08T09:46:55.115 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2023-10-08T09:46:55.116 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression: NoCompression 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.num_levels: 7 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T09:46:55.117 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T09:46:55.118 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T09:46:55.119 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T09:46:55.120 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T09:46:55.121 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.table_properties_collectors: 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.inplace_update_support: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.bloom_locality: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.max_successive_merges: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T09:46:55.122 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.ttl: 2592000 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enable_blob_files: false 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.min_blob_size: 0 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T09:46:55.123 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 664b29f9-caa9-4e9d-a0b9-86aecb751173 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758415032351, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758415033293, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85350, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 272, "table_properties": {"data_size": 83453, "index_size": 243, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11217, "raw_average_key_size": 48, "raw_value_size": 77139, "raw_average_value_size": 333, "num_data_blocks": 10, "num_entries": 231, "num_filter_entries": 231, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696758415, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "664b29f9-caa9-4e9d-a0b9-86aecb751173", "db_session_id": "H9DKPC4UT4XYAHIL3YNS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758415033362, "job": 1, "event": "recovery_finished"} 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ba0e626000 2023-10-08T09:46:55.124 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: DB pointer 0x55ba0e612000 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2023-10-08T09:46:55.125 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 85.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 85.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 128.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T09:46:55.126 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 24.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 24.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x55ba0e5891f0#7 capacity: 512.00 MB usage: 1.27 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.127 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???) e1 preinit fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).mds e1 new map 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).mds e1 print_map 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T09:46:55.128 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:46:55.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:46:55.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:46:55.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-08T09:46:55.129 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 systemd[1]: Started Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:46:55.129 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-08T09:46:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:46:55.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: monmap e1: 1 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-10-08T09:46:55.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: fsmap 2023-10-08T09:46:55.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T09:46:55.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:55 smithi097 ceph-mon[99491]: mgrmap e1: no daemons active 2023-10-08T09:46:55.953 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-08T09:46:55.986 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:46:55.987 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2023-10-08T09:46:55.987 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-08T09:46:55.987 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-08T09:46:56.169 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T09:46:56.169 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-08T09:46:56.258 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7.target.wants/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@.service. 2023-10-08T09:46:56.556 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 systemd[1]: Starting Ceph mgr.x for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:46:56.848 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3262308877' entity='client.admin' 2023-10-08T09:46:56.848 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 podman[99718]: 2023-10-08T09:46:56.848 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 bash[99718]: 931644c8953d654cc660ba33434cdd836810ec2051d5c800d4b98c9775a4a4e8 2023-10-08T09:46:56.860 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:46:56.860 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2023-10-08T09:46:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:46:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:46:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2023-10-08T09:46:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T09:46:56.875 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:46:56.875 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2023-10-08T09:46:56.883 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:46:56.883 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:46:56.883 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2023-10-08T09:46:56.884 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-08T09:46:56.884 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2023-10-08T09:46:56.884 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2023-10-08T09:46:57.136 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 systemd[1]: Started Ceph mgr.x for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:46:57.136 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:56.907+0000 7f35eeae6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:46:57.136 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:56 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:56.991+0000 7f35eeae6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:46:57.136 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.083+0000 7f35eeae6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:46:57.452 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.186+0000 7f35eeae6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:46:57.453 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.363+0000 7f35eeae6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:46:57.748 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3088403858' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:46:57.829 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:46:57.832 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:46:57.832 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:46:57.832 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:46:57.832 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:46:57.832 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-10-08T09:46:57.833 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:46:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:46:57.837 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:46:57.838 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:46:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:46:49.064314+0000", 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:46:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2023-10-08T09:46:58.065 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.746+0000 7f35eeae6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:46:58.065 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.880+0000 7f35eeae6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:46:58.065 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:57 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:57.969+0000 7f35eeae6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:46:58.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:58 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:58.063+0000 7f35eeae6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:46:58.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:58 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:58.479+0000 7f35eeae6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:46:59.452 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:59 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:59.151+0000 7f35eeae6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:46:59.452 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:46:59 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:46:59.232+0000 7f35eeae6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:47:00.286 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.007+0000 7f35eeae6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:00.286 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.105+0000 7f35eeae6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:47:00.590 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.284+0000 7f35eeae6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:47:00.780 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-10-08T09:47:00.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:47:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:47:00.786 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:47:00.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:46:49.064314+0000", 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:47:00.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:00.789 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2023-10-08T09:47:00.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.587+0000 7f35eeae6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:47:00.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.818+0000 7f35eeae6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:47:01.228 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:00.965+0000 7f35eeae6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:47:01.229 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2825731796' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:47:01.702 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:01 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:01.226+0000 7f35eeae6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:47:02.202 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:01 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:01.918+0000 7f35eeae6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:47:02.202 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:02 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:02.047+0000 7f35eeae6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:47:02.570 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:02 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:02.240+0000 7f35eeae6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:47:02.571 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:02 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:02.353+0000 7f35eeae6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:47:03.316 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:02 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:02.568+0000 7f35eeae6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:47:03.628 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:03.629 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:47:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:47:03.633 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:47:03.633 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:47:03.633 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.633 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:47:03.635 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:03.637 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:46:49.064314+0000", 2023-10-08T09:47:03.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:03.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:03.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:47:03.638 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:03.638 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2023-10-08T09:47:03.647 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2546610967' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:47:03.648 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:03 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:03.469+0000 7f35eeae6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:47:04.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: Activating manager daemon x 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: mgrmap e2: x(active, starting, since 0.00304252s) 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: Manager daemon x is now available 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:47:04.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' 2023-10-08T09:47:04.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' 2023-10-08T09:47:04.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:04 smithi097 ceph-mon[99491]: from='mgr.14100 172.21.15.97:0/3100412443' entity='mgr.x' 2023-10-08T09:47:05.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:05 smithi097 ceph-mon[99491]: mgrmap e3: x(active, since 1.00666s) 2023-10-08T09:47:07.943 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:07 smithi097 ceph-mon[99491]: mgrmap e4: x(active, since 3s) 2023-10-08T09:47:08.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:08 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/597721356' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:47:11.090 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:47:11.091 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:11.092 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:47:11.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:11.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:46:49.064314+0000", 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:11.096 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2023-10-08T09:47:13.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:13 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/741487812' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T09:47:15.043 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T09:47:15.044 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2023-10-08T09:47:16.380 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/161250754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-08T09:47:16.380 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: ignoring --setuser ceph since I am not root 2023-10-08T09:47:16.380 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: ignoring --setgroup ceph since I am not root 2023-10-08T09:47:16.381 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:16.208+0000 7f4d80084200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:47:16.381 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:16.289+0000 7f4d80084200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:47:16.381 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:16.379+0000 7f4d80084200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:47:16.636 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:16.481+0000 7f4d80084200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:47:16.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:16 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:16.664+0000 7f4d80084200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:47:17.300 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:17.053+0000 7f4d80084200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:47:17.300 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:17.192+0000 7f4d80084200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:47:17.301 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:17.288+0000 7f4d80084200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:47:17.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/161250754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-08T09:47:17.301 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-mon[99491]: mgrmap e5: x(active, since 12s) 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2023-10-08T09:47:17.531 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2023-10-08T09:47:17.702 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:17.386+0000 7f4d80084200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:47:18.046 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:17 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:17.836+0000 7f4d80084200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:47:18.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/461571991' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T09:47:18.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:18 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:18.497+0000 7f4d80084200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:47:18.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:18 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:18.578+0000 7f4d80084200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:47:19.612 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:19 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:19.345+0000 7f4d80084200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:20.147 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:19 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:19.437+0000 7f4d80084200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:47:20.148 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:19 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:19.611+0000 7f4d80084200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:20.202 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:19 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:19.888+0000 7f4d80084200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:47:20.322 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:20 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:20.109+0000 7f4d80084200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:47:20.517 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:20 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:20.256+0000 7f4d80084200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:47:23.399 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:20 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:20.516+0000 7f4d80084200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:47:23.399 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:21.207+0000 7f4d80084200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:47:23.399 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:21.337+0000 7f4d80084200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:47:23.399 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:21.529+0000 7f4d80084200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:21.642+0000 7f4d80084200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:21.859+0000 7f4d80084200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:22.710+0000 7f4d80084200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: Active manager daemon x restarted 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: Activating manager daemon x 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: osdmap e2: 0 total, 0 up, 0 in 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: mgrmap e6: x(active, starting, since 0.00319316s) 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:47:23.400 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:47:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:47:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:47:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: Manager daemon x is now available 2023-10-08T09:47:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:23.401 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:23.403 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:23.403 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:23.403 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:22 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:24.094 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:24.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-08T09:47:24.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T09:47:24.095 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:24.095 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2023-10-08T09:47:24.095 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2023-10-08T09:47:24.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: Found migration_current of "None". Setting to last migration. 2023-10-08T09:47:24.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:47:24.107 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:47:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:24.108 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:23 smithi097 ceph-mon[99491]: mgrmap e7: x(active, since 1.00529s) 2023-10-08T09:47:25.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:24 smithi097 ceph-mon[99491]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-08T09:47:25.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:24 smithi097 ceph-mon[99491]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-08T09:47:25.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:24 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:25.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:24 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:25.153 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:24 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:26.134 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:24] ENGINE Bus STARTING 2023-10-08T09:47:26.134 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:24] ENGINE Serving on https://172.21.15.97:7150 2023-10-08T09:47:26.134 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:24] ENGINE Serving on http://172.21.15.97:8765 2023-10-08T09:47:26.134 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:24] ENGINE Bus STARTED 2023-10-08T09:47:26.135 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:26.135 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:26 smithi097 ceph-mon[99491]: mgrmap e8: x(active, since 3s) 2023-10-08T09:47:26.178 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-08T09:47:26.178 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2023-10-08T09:47:27.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-mon[99491]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:28 smithi097 ceph-mon[99491]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:28 smithi097 ceph-mon[99491]: Generating ssh key... 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:28 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:28 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: Generating public/private rsa key pair. 2023-10-08T09:47:28.313 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: Your identification has been saved in /tmp/tmppyna92m3/key. 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: Your public key has been saved in /tmp/tmppyna92m3/key.pub. 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: The key fingerprint is: 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: SHA256:yAYAXPPh3xF43hZsn3wGnK/ZSfbEFZpvI4ESHb+CXj0 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: The key's randomart image is: 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: +---[RSA 3072]----+ 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: |o.oo . .oo.o ...| 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | . .+ .. .o=o+o .| 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | .o ooo.+=+..| 2023-10-08T09:47:28.314 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | o....+o.++*o| 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | +.So.o EO=o| 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | . . . .o+oo| 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | . | 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | | 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: | | 2023-10-08T09:47:28.315 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: +----[SHA256]-----+ 2023-10-08T09:47:29.300 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:29 smithi097 ceph-mon[99491]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:31.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:31.266 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-08T09:47:31.266 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2023-10-08T09:47:31.267 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2023-10-08T09:47:31.267 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:31 smithi097 ceph-mon[99491]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:32.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:32 smithi097 ceph-mon[99491]: Deploying cephadm binary to smithi097 2023-10-08T09:47:32.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2023-10-08T09:47:32.806 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2023-10-08T09:47:33.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:33 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:33.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:33 smithi097 ceph-mon[99491]: Added host smithi097 2023-10-08T09:47:33.661 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:33 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:34.259 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-08T09:47:34.259 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2023-10-08T09:47:34.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:34 smithi097 ceph-mon[99491]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:34.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:34 smithi097 ceph-mon[99491]: Saving service mon spec with placement count:5 2023-10-08T09:47:34.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:34 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:34.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:34 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:35.403 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-08T09:47:36.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:35 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:36.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:35 smithi097 ceph-mon[99491]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:36.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:35 smithi097 ceph-mon[99491]: Saving service mgr spec with placement count:2 2023-10-08T09:47:36.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:35 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:37.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2520624251' entity='client.admin' 2023-10-08T09:47:37.693 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2023-10-08T09:47:38.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:38 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:38.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2543895453' entity='client.admin' 2023-10-08T09:47:38.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:38 smithi097 ceph-mon[99491]: from='mgr.14120 172.21.15.97:0/3229212051' entity='mgr.x' 2023-10-08T09:47:39.377 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2720324019' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-08T09:47:39.377 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: ignoring --setuser ceph since I am not root 2023-10-08T09:47:39.377 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: ignoring --setgroup ceph since I am not root 2023-10-08T09:47:39.377 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:39.296+0000 7fc5fb5fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:47:39.377 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:39.376+0000 7fc5fb5fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:47:39.629 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:39.466+0000 7fc5fb5fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:47:39.629 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:39.561+0000 7fc5fb5fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:47:39.881 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:39 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:39.741+0000 7fc5fb5fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:47:40.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2720324019' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-08T09:47:40.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-mon[99491]: mgrmap e9: x(active, since 16s) 2023-10-08T09:47:40.203 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:40.126+0000 7fc5fb5fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:47:40.702 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:40.263+0000 7fc5fb5fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:47:40.703 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:40.355+0000 7fc5fb5fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:47:40.703 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:40.454+0000 7fc5fb5fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:47:40.983 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2023-10-08T09:47:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2023-10-08T09:47:41.115 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:40 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:40.885+0000 7fc5fb5fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:47:41.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:41 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/312596150' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T09:47:41.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:41 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:41.577+0000 7fc5fb5fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:47:41.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:41 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:41.659+0000 7fc5fb5fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:47:42.687 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:42 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:42.424+0000 7fc5fb5fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:42.688 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:42 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:42.515+0000 7fc5fb5fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:47:42.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:42 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:42.686+0000 7fc5fb5fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:47:43.333 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:42 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:42.964+0000 7fc5fb5fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:47:43.333 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:43 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:43.186+0000 7fc5fb5fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:47:43.952 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:43 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:43.332+0000 7fc5fb5fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:47:43.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:43 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:43.594+0000 7fc5fb5fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:47:44.609 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:44 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:44.286+0000 7fc5fb5fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:47:44.609 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:44 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:44.416+0000 7fc5fb5fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:47:44.936 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:44 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:44.608+0000 7fc5fb5fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:47:44.936 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:44 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:44.720+0000 7fc5fb5fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:47:45.202 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:44 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:44.935+0000 7fc5fb5fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:47:46.046 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 09:47:45 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T09:47:45.783+0000 7fc5fb5fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:47:46.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: Active manager daemon x restarted 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: Activating manager daemon x 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: osdmap e3: 0 total, 0 up, 0 in 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: mgrmap e10: x(active, starting, since 0.00370803s) 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: Manager daemon x is now available 2023-10-08T09:47:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:47:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:47:47.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:47:47.148 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-08T09:47:47.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T09:47:47.149 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:47:47.149 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2023-10-08T09:47:47.149 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2023-10-08T09:47:48.126 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:46] ENGINE Bus STARTING 2023-10-08T09:47:48.126 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:46] ENGINE Serving on http://172.21.15.97:8765 2023-10-08T09:47:48.126 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-08T09:47:48.127 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: mgrmap e11: x(active, since 1.00554s) 2023-10-08T09:47:48.127 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-08T09:47:48.127 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:46] ENGINE Serving on https://172.21.15.97:7150 2023-10-08T09:47:48.127 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: [08/Oct/2023:09:47:46] ENGINE Bus STARTED 2023-10-08T09:47:48.127 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:48.441 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-08T09:47:48.441 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2023-10-08T09:47:49.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:49 smithi097 ceph-mon[99491]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:49.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:49.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:49.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:49.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:49 smithi097 ceph-mon[99491]: mgrmap e12: x(active, since 3s) 2023-10-08T09:47:49.781 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UFa2yPdoodN6xheOIdq2f.gCy8paWaSANEE4QbZmRIRm0jZg3q7m6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696758469, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-08T09:47:49.782 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2023-10-08T09:47:50.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:50 smithi097 ceph-mon[99491]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:50.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:50.923 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2023-10-08T09:47:50.934 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:47:50.934 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2023-10-08T09:47:50.941 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:47:50.941 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2023-10-08T09:47:50.942 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2023-10-08T09:47:50.942 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-08T09:47:50.943 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2023-10-08T09:47:50.943 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:50.943 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097.front.sepia.ceph.com:8443/ 2023-10-08T09:47:50.943 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2023-10-08T09:47:50.944 INFO:teuthology.orchestra.run.smithi097.stdout: Password: h2manoeiur 2023-10-08T09:47:50.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:50.944 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config directory 2023-10-08T09:47:50.944 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2023-10-08T09:47:51.173 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:51 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/444188431' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-08T09:47:53.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:47:53.236 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2023-10-08T09:47:53.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:53 smithi097 ceph-mon[99491]: mgrmap e13: x(active, since 6s) 2023-10-08T09:47:53.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:53 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/641846091' entity='client.admin' 2023-10-08T09:47:53.268 INFO:tasks.cephadm:Fetching config... 2023-10-08T09:47:53.268 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:47:53.268 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-08T09:47:53.286 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-08T09:47:53.287 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:47:53.287 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-08T09:47:53.346 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-08T09:47:53.346 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:47:53.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-08T09:47:53.414 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-08T09:47:53.414 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:47:53.414 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-08T09:47:53.471 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-08T09:47:53.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:47:53.551 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:53.564 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:47:53.608 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:53.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:47:53.670 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk+M6GLaDcajtiqRWsrWkAOsjVWo4kVDTz/rwWKcvrUe/cf27VFuVBkzbwsy/Qihxem/PfnwTTxvykj5NMXfTChOU4iTf/ZvnSv/WFLQYY3sd+5wlzYp7USEwCbKfWSDh3n/wEz7QaXHvNRVzmWC6TCXYwY7ege09+T15FR8B41ZttCfbI71acIMfblBgLADwiUroznBLt8iKJa5jbTDl4v39v7vj1IOGxGoXU8xBqnBaJTYMtPKnhEl560+do4JxkEglfziSINOklf5T4bT7tkj5iMR9ybZCbczZ6PizZMk8WWb7JKEZZJOd8SnOI8eoaUFneqyV6JqXSNzOOMkOnpvyorGwkeN7G9RAwvy1v6ePgazBJGtFGERwmVI3zTstIbbrE2SP3HWEmV5eR3nN2S0Lu/ifdy9Zq4UlxukG0jbb6ZXtbSIC8xImD4rapCmUhGSOhSgQgjH/vOssQuGfcVVXcOsROoDQXIpLNNNxB3Qb3G/B5kBO7lfdPGrMD9rk= ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:47:53.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-08T09:47:54.262 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:47:56.377 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:56.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:56.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:47:56.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:56.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:56.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:56.475 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-08T09:47:56.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-08T09:47:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:47:57.284 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1587555999' entity='client.admin' 2023-10-08T09:47:57.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:57.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:58.898 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2023-10-08T09:47:58.898 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:47:58.898 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2023-10-08T09:47:58.918 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:47:58.918 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:47:58.977 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2023-10-08T09:47:58.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi178 2023-10-08T09:47:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:47:59.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: Updating smithi097:/etc/ceph/ceph.conf 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: Updating smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: Updating smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:47:59.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:47:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:01.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:01 smithi097 ceph-mon[99491]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:02.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:02 smithi097 ceph-mon[99491]: Deploying cephadm binary to smithi178 2023-10-08T09:48:03.369 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi178' with addr '172.21.15.178' 2023-10-08T09:48:03.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T09:48:04.257 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:04.633 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:04 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:04.633 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:04 smithi097 ceph-mon[99491]: Added host smithi178 2023-10-08T09:48:04.633 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:04 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:04.633 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:04 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:05.665 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:48:05.666 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2023-10-08T09:48:05.686 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:05 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:06.011 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2023-10-08T09:48:06.011 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:48:06.011 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2023-10-08T09:48:06.032 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:48:06.032 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:06.091 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2023-10-08T09:48:06.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi186 2023-10-08T09:48:06.646 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:07.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:06 smithi097 ceph-mon[99491]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:48:07.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:06 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:07.056 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:06 smithi097 ceph-mon[99491]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:09.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:09.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:09.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:09.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:09.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:09.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:09.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:48:09.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:09.205 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:10.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:09 smithi097 ceph-mon[99491]: Deploying cephadm binary to smithi186 2023-10-08T09:48:10.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:09 smithi097 ceph-mon[99491]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-08T09:48:10.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:09 smithi097 ceph-mon[99491]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:10.423 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi186' with addr '172.21.15.186' 2023-10-08T09:48:10.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T09:48:11.351 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:11.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:11.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:12.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:12 smithi097 ceph-mon[99491]: Added host smithi186 2023-10-08T09:48:12.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:12.769 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:48:12.769 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2023-10-08T09:48:13.205 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-08T09:48:13.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-08T09:48:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:13 smithi097 ceph-mon[99491]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:13.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:13.758 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:14.243 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:14 smithi097 ceph-mon[99491]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:48:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:15 smithi097 ceph-mon[99491]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2023-10-08T09:48:16.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2791544055' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T09:48:16.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:16.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:16.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:16.414 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:16.414 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:48:16.414 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:16.414 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:16.415 INFO:tasks.cephadm:Adding mon.a on smithi097 2023-10-08T09:48:16.415 INFO:tasks.cephadm:Adding mon.b on smithi178 2023-10-08T09:48:16.415 INFO:tasks.cephadm:Adding mon.c on smithi186 2023-10-08T09:48:16.415 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c' 2023-10-08T09:48:16.999 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2791544055' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:18.399 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2023-10-08T09:48:18.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:18 smithi097 ceph-mon[99491]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:18 smithi097 ceph-mon[99491]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:48:18.766 DEBUG:teuthology.orchestra.run.smithi178:mon.b> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-08T09:48:18.769 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-08T09:48:18.771 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:48:18.771 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:48:18.800 INFO:journalctl@ceph.mon.c.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:48:18.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:48:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:19.749 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:20.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:20 smithi097 ceph-mon[99491]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:20.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:20 smithi097 ceph-mon[99491]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c;count:3 2023-10-08T09:48:20.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:20 smithi097 ceph-mon[99491]: Deploying daemon mon.c on smithi186 2023-10-08T09:48:21.440 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:48:21.440 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:46:47.004048Z","created":"2023-10-08T09:46:47.004048Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-08T09:48:21.441 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2023-10-08T09:48:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:21 smithi097 ceph-mon[99491]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:22.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:22 smithi186 ceph-mon[104393]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T09:48:22.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:22 smithi186 systemd[1]: Started Ceph mon.c for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:48:22.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:22 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/2384546283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:22.798 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:48:22.799 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:48:23.365 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: mon.a calling monitor election 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: mon.c calling monitor election 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: fsmap 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: mgrmap e13: x(active, since 41s) 2023-10-08T09:48:27.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: overall HEALTH_OK 2023-10-08T09:48:27.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:27.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:27.700 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:48:27.701 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:48:22.120157Z","created":"2023-10-08T09:46:47.004048Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-08T09:48:27.703 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2023-10-08T09:48:27.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: mon.a calling monitor election 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: mon.c calling monitor election 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: fsmap 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: mgrmap e13: x(active, since 41s) 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: overall HEALTH_OK 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:27.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:28.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:28 smithi186 ceph-mon[104393]: Deploying daemon mon.b on smithi178 2023-10-08T09:48:28.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:28 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/3834234386' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:28.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:28 smithi186 ceph-mon[104393]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:28.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:28 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:28.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:28 smithi097 ceph-mon[99491]: Deploying daemon mon.b on smithi178 2023-10-08T09:48:28.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:28 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/3834234386' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:28.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:28 smithi097 ceph-mon[99491]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:28.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:28 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:29.015 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:48:29.016 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:48:29.586 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:48:29.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:29 smithi178 systemd[1]: Starting Ceph mon.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:48:30.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:29 smithi178 podman[103115]: 2023-10-08T09:48:30.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T09:48:30.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 8 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pidfile_write: ignore empty --pid-file 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 bash[103115]: 38e4b0bdddcaf5cd7e9dfd8a05ba1efac7c2ac25d829dc828c14f5d590143288 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: load: jerasure load: lrc 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: RocksDB version: 7.9.2 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Git sha 0 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: DB SUMMARY 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: DB Session ID: UKHR0846VFBKX1ZHMCDH 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: CURRENT file: CURRENT 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-08T09:48:30.250 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.error_if_exists: 0 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.create_if_missing: 0 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.paranoid_checks: 1 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.env: 0x558d4369eda0 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.info_log: 0x558d43fd9640 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T09:48:30.251 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.statistics: (nil) 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.use_fsync: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_log_file_size: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_fallocate: 1 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.use_direct_reads: 0 2023-10-08T09:48:30.252 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.db_log_dir: 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.wal_dir: 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.write_buffer_manager: 0x558d43f91360 2023-10-08T09:48:30.253 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.unordered_write: 0 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T09:48:30.254 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.row_cache: None 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.wal_filter: None 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.two_write_queues: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.wal_compression: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.atomic_flush: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.log_readahead_size: 0 2023-10-08T09:48:30.255 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_background_jobs: 2 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_background_compactions: -1 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_subcompactions: 1 2023-10-08T09:48:30.256 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_open_files: -1 2023-10-08T09:48:30.257 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_background_flushes: -1 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Compression algorithms supported: 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kZSTD supported: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kXpressCompression supported: 0 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kZlibCompression supported: 1 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kSnappyCompression supported: 1 2023-10-08T09:48:30.258 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kLZ4Compression supported: 1 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: kBZip2Compression supported: 0 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.merge_operator: 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_filter: None 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T09:48:30.259 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558d43fd9740) 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks: 1 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_top_level_index_and_filter: 1 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_type: 0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_index_type: 0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_shortening: 1 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: checksum: 4 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: no_block_cache: 0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache: 0x558d4402f1f0 2023-10-08T09:48:30.260 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_name: BinnedLRUCache 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_options: 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: capacity : 536870912 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: num_shard_bits : 4 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: strict_capacity_limit : 0 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: high_pri_pool_ratio: 0.000 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_compressed: (nil) 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: persistent_cache: (nil) 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size: 4096 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size_deviation: 10 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_restart_interval: 16 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_block_restart_interval: 1 2023-10-08T09:48:30.261 INFO:journalctl@ceph.mon.b.smithi178.stdout: metadata_block_size: 4096 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: partition_filters: 0 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: use_delta_encoding: 1 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: filter_policy: bloomfilter 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: whole_key_filtering: 1 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: verify_compression: 0 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: read_amp_bytes_per_bit: 0 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: format_version: 5 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_index_compression: 1 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_align: 0 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: max_auto_readahead_size: 262144 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: prepopulate_block_cache: 0 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: initial_auto_readahead_size: 8192 2023-10-08T09:48:30.262 INFO:journalctl@ceph.mon.b.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression: NoCompression 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.num_levels: 7 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T09:48:30.263 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T09:48:30.265 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T09:48:30.266 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T09:48:30.267 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T09:48:30.268 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.table_properties_collectors: 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.inplace_update_support: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.bloom_locality: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.max_successive_merges: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T09:48:30.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.ttl: 2592000 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enable_blob_files: false 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.min_blob_size: 0 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T09:48:30.270 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4e54ea32-8ef3-4dcd-af40-f63673fc8093 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758510158589, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758510158991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696758510, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4e54ea32-8ef3-4dcd-af40-f63673fc8093", "db_session_id": "UKHR0846VFBKX1ZHMCDH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696758510159058, "job": 1, "event": "recovery_finished"} 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558d440cc000 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: DB pointer 0x558d440b8000 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T09:48:30.271 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** DB Stats ** 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-08T09:48:30.272 INFO:journalctl@ceph.mon.b.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T09:48:30.273 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: Block cache BinnedLRUCache@0x558d4402f1f0#8 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T09:48:30.274 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: starting mon.b rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(???) e0 preinit fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).mds e1 new map 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: e1 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: legacy client fscid: -1 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2023-10-08T09:48:30.275 INFO:journalctl@ceph.mon.b.smithi178.stdout: No filesystems configured 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Added host smithi186 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2791544055' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2791544055' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:30.279 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c;count:3 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Deploying daemon mon.c on smithi186 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/2384546283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.a calling monitor election 2023-10-08T09:48:30.280 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.c calling monitor election 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: monmap e2: 2 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:30.281 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: fsmap 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mgrmap e13: x(active, since 41s) 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: overall HEALTH_OK 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: Deploying daemon mon.b on smithi178 2023-10-08T09:48:30.282 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/3834234386' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:30.283 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:30.283 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:30.283 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 ceph-mon[103156]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T09:48:30.283 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:30 smithi178 systemd[1]: Started Ceph mon.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:48:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: mon.a calling monitor election 2023-10-08T09:48:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: mon.c calling monitor election 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: mon.b calling monitor election 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:48:35.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: fsmap 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: mgrmap e13: x(active, since 49s) 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: overall HEALTH_OK 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: mon.a calling monitor election 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: mon.c calling monitor election 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: mon.b calling monitor election 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: fsmap 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: mgrmap e13: x(active, since 49s) 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: overall HEALTH_OK 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:35.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: mon.a calling monitor election 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: mon.c calling monitor election 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: mon.b calling monitor election 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: monmap e3: 3 mons at {a=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: fsmap 2023-10-08T09:48:35.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:48:35.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: mgrmap e13: x(active, since 49s) 2023-10-08T09:48:35.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: overall HEALTH_OK 2023-10-08T09:48:35.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:35.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:36.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:48:37.024 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-08T09:48:37.024 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:48:30.181232Z","created":"2023-10-08T09:46:47.004048Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-10-08T09:48:37.027 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2023-10-08T09:48:37.443 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-08T09:48:37.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi097:/etc/ceph/ceph.conf 2023-10-08T09:48:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/1517249535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.038 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi097:/etc/ceph/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/1517249535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.039 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:38.040 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:38.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi097:/etc/ceph/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/1517249535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: Updating smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:38.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:38 smithi097 ceph-mon[99491]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:48:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:38 smithi097 ceph-mon[99491]: Reconfiguring daemon mon.a on smithi097 2023-10-08T09:48:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:38 smithi097 ceph-mon[99491]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:38 smithi186 ceph-mon[104393]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:48:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:38 smithi186 ceph-mon[104393]: Reconfiguring daemon mon.a on smithi097 2023-10-08T09:48:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:38 smithi186 ceph-mon[104393]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:38 smithi178 ceph-mon[103156]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:48:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:38 smithi178 ceph-mon[103156]: Reconfiguring daemon mon.a on smithi097 2023-10-08T09:48:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:38 smithi178 ceph-mon[103156]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:39.638 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:48:39.638 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-10-08T09:48:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:48:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-10-08T09:48:39.911 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:48:39.911 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:39.911 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:39.912 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:39.912 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: Reconfiguring daemon mon.b on smithi178 2023-10-08T09:48:39.912 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1160887959' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:39.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:48:39.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: Reconfiguring daemon mon.b on smithi178 2023-10-08T09:48:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1160887959' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:40.017 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-08T09:48:40.017 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:48:40.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: Reconfiguring daemon mon.b on smithi178 2023-10-08T09:48:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1160887959' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:40.048 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:48:40.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:40.115 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:48:40.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:48:40.147 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:48:40.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:40.216 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:48:40.217 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:48:40.253 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:48:40.253 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:48:40.322 INFO:tasks.cephadm:Adding mgr.x on smithi097 2023-10-08T09:48:40.322 INFO:tasks.cephadm:Adding mgr.y on smithi178 2023-10-08T09:48:40.323 INFO:tasks.cephadm:Adding mgr.z on smithi186 2023-10-08T09:48:40.323 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi097=x;smithi178=y;smithi186=z' 2023-10-08T09:48:41.284 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:41.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:41.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:41.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:48:41.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:48:41.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:42.328 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: Reconfiguring daemon mon.c on smithi186 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:42.329 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: Reconfiguring daemon mon.c on smithi186 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: Reconfiguring daemon mon.c on smithi186 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:42.676 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2023-10-08T09:48:43.134 DEBUG:teuthology.orchestra.run.smithi178:mgr.y> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-08T09:48:43.137 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-08T09:48:43.139 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-08T09:48:43.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-08T09:48:43.141 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-08T09:48:43.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-08T09:48:43.168 INFO:journalctl@ceph.mgr.y.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:48:43.170 INFO:journalctl@ceph.mgr.z.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:48:43.184 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-08T09:48:43.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-08T09:48:43.227 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-08T09:48:43.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-08T09:48:43.270 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-08T09:48:43.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-08T09:48:43.313 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-08T09:48:43.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-08T09:48:43.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:43.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:43.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:43.339 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.340 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-08T09:48:43.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:43.347 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:43.348 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:43.348 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.382 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-08T09:48:43.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-08T09:48:43.428 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-08T09:48:43.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-08T09:48:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:43.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:43.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:43.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:43.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:43.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:43.470 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-08T09:48:43.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-08T09:48:43.514 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-08T09:48:43.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-08T09:48:43.559 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-08T09:48:43.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-08T09:48:43.609 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-08T09:48:43.610 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-08T09:48:43.661 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-08T09:48:43.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-08T09:48:43.662 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:43.716 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-08T09:48:43.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-08T09:48:43.770 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-08T09:48:43.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-08T09:48:43.813 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-08T09:48:43.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-08T09:48:43.857 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-08T09:48:43.857 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-08T09:48:43.925 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-08T09:48:43.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-08T09:48:43.979 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-08T09:48:43.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-08T09:48:44.023 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-08T09:48:44.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-08T09:48:44.104 DEBUG:tasks.cephadm:set 21 configs 2023-10-08T09:48:44.104 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi097') 2023-10-08T09:48:44.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-mon[104393]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi178=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:44.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-mon[104393]: Saving service mgr spec with placement smithi097=x;smithi178=y;smithi186=z;count:3 2023-10-08T09:48:44.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-mon[104393]: Deploying daemon mgr.z on smithi186 2023-10-08T09:48:44.276 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 systemd[1]: Starting Ceph mgr.z for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:48:44.368 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:44 smithi097 ceph-mon[99491]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi178=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:44.369 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:44 smithi097 ceph-mon[99491]: Saving service mgr spec with placement smithi097=x;smithi178=y;smithi186=z;count:3 2023-10-08T09:48:44.369 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:44 smithi097 ceph-mon[99491]: Deploying daemon mgr.z on smithi186 2023-10-08T09:48:44.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:44 smithi178 ceph-mon[103156]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi178=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:48:44.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:44 smithi178 ceph-mon[103156]: Saving service mgr spec with placement smithi097=x;smithi178=y;smithi186=z;count:3 2023-10-08T09:48:44.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:44 smithi178 ceph-mon[103156]: Deploying daemon mgr.z on smithi186 2023-10-08T09:48:44.456 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:44.465 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:44.612 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:44.717 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 podman[106255]: 2023-10-08T09:48:44.717 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 bash[106255]: 5a9c483f2de43ddce3186b43e1e158f6ed6bfe7ca6520cc506513a9b0c5edbe7 2023-10-08T09:48:44.717 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-mgr[106295]: -- 172.21.15.186:0/1982334870 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562e531b11e0 con 0x562e53186c00 2023-10-08T09:48:45.000 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 systemd[1]: Started Ceph mgr.z for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:48:45.000 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:44.733+0000 7fb2b1541200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:48:45.000 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:44.817+0000 7fb2b1541200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:48:45.001 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:44 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:44.907+0000 7fb2b1541200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:48:45.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:45.259 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:44.999+0000 7fb2b1541200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:45.171+0000 7fb2b1541200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:45.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:45.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:45.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:45.291 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:45.292 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:45.338 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:45.424 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:48:45.425 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:45.425 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:45.640 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:45.674 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:45.850 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:45.535+0000 7fb2b1541200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:48:45.850 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:45.667+0000 7fb2b1541200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:48:45.850 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:45.755+0000 7fb2b1541200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:48:46.120 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:45 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:45.849+0000 7fb2b1541200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:48:46.358 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:46 smithi178 ceph-mon[103156]: Deploying daemon mgr.y on smithi178 2023-10-08T09:48:46.358 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:46 smithi178 systemd[1]: Starting Ceph mgr.y for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:48:46.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:46 smithi097 ceph-mon[99491]: Deploying daemon mgr.y on smithi178 2023-10-08T09:48:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:46 smithi186 ceph-mon[104393]: Deploying daemon mgr.y on smithi178 2023-10-08T09:48:46.525 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:46 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:46.271+0000 7fb2b1541200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:48:46.917 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:46 smithi178 podman[104357]: 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3749344186' entity='client.admin' 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3749344186' entity='client.admin' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.205 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:46 smithi178 bash[104357]: 56e509f49bff314bf4aa577c2f9b74af8aeb86c9eda79e44bbd977206ceef2c8 2023-10-08T09:48:47.206 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:46 smithi178 ceph-mgr[104397]: -- 172.21.15.178:0/792169831 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585cbd731e0 con 0x5585cbd48c00 2023-10-08T09:48:47.206 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:46 smithi178 systemd[1]: Started Ceph mgr.y for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:48:47.206 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.047+0000 7fd528ec5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:48:47.206 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.132+0000 7fd528ec5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:46 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:46.941+0000 7fb2b1541200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:47.023+0000 7fb2b1541200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3749344186' entity='client.admin' 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:47.510 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.225+0000 7fd528ec5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:48:47.510 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.326+0000 7fd528ec5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:48:47.747 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:47.895 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.509+0000 7fd528ec5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:48:48.120 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:47.844+0000 7fb2b1541200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:48:48.120 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:47 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:47.937+0000 7fb2b1541200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:48:48.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:48.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:48.205 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:47 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:47.895+0000 7fd528ec5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:48:48.205 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:48 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:48.029+0000 7fd528ec5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:48:48.205 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:48 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:48.119+0000 7fd528ec5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:48:48.419 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:48.419 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:48 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:48.119+0000 7fb2b1541200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:48:48.419 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:48 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:48.417+0000 7fb2b1541200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:48:48.442 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:48.445 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:48.456 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:48.683 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:48 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:48.224+0000 7fd528ec5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:48:48.775 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:48 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:48.658+0000 7fb2b1541200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:48:48.955 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:48 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:48.683+0000 7fd528ec5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:48:49.079 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:48 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:48.808+0000 7fb2b1541200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:48:49.099 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi097') 2023-10-08T09:48:49.377 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1503319596' entity='client.admin' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:49.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:49.353+0000 7fd528ec5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1503319596' entity='client.admin' 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.435 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.436 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:49.436 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.436 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:49.436 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:49.436 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.525 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:49.078+0000 7fb2b1541200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:48:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1503319596' entity='client.admin' 2023-10-08T09:48:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:48:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:49.554 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi097') 2023-10-08T09:48:49.705 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:49 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:49.434+0000 7fd528ec5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:48:50.112 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:49.784+0000 7fb2b1541200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:48:50.112 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:49 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:49.916+0000 7fb2b1541200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:48:50.112 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:50 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:50.111+0000 7fb2b1541200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:48:50.447 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:50 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:50.228+0000 7fb2b1541200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:48:50.447 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:50 smithi186 ceph-mon[104393]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:48:50.447 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:50 smithi186 ceph-mon[104393]: Reconfiguring daemon mgr.x on smithi097 2023-10-08T09:48:50.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:50 smithi097 ceph-mon[99491]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:48:50.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:50 smithi097 ceph-mon[99491]: Reconfiguring daemon mgr.x on smithi097 2023-10-08T09:48:50.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-mon[103156]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:48:50.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-mon[103156]: Reconfiguring daemon mgr.x on smithi097 2023-10-08T09:48:50.455 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:50.203+0000 7fd528ec5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:48:50.455 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:50.294+0000 7fd528ec5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:48:50.566 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.720+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd958038af0 msgr2=0x7fd95803afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.720+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd958038af0 0x7fd95803afb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd964009550 tx=0x7fd9640091b0 comp rx=0 tx=0).stop 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd96c100930 msgr2=0x7fd96c19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd96c100930 0x7fd96c19b2b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b580 tx=0x7fd95400f690 comp rx=0 tx=0).stop 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 shutdown_connections 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd958038af0 0x7fd95803afb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd96c1012d0 0x7fd96c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd96c0ff730 0x7fd96c068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 --2- 172.21.15.97:0/763375527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd96c100930 0x7fd96c19b2b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 >> 172.21.15.97:0/763375527 conn(0x7fd96c0facf0 msgr2=0x7fd96c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 shutdown_connections 2023-10-08T09:48:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:50.721+0000 7fd972dcb700 1 -- 172.21.15.97:0/763375527 wait complete. 2023-10-08T09:48:50.746 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:50.467+0000 7fd528ec5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:48:51.118 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:50.745+0000 7fd528ec5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:48:51.118 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:50.972+0000 7fd528ec5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:48:51.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:51 smithi097 ceph-mon[99491]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:51 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4269904558' entity='client.admin' 2023-10-08T09:48:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:51 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/763375527' entity='client.admin' 2023-10-08T09:48:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:51.379 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:51.117+0000 7fd528ec5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:48:51.379 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-mon[103156]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:51.380 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4269904558' entity='client.admin' 2023-10-08T09:48:51.380 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/763375527' entity='client.admin' 2023-10-08T09:48:51.380 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:51 smithi186 ceph-mon[104393]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:51 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4269904558' entity='client.admin' 2023-10-08T09:48:51.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:51 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/763375527' entity='client.admin' 2023-10-08T09:48:51.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:51.526 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:50 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:50.446+0000 7fb2b1541200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:48:51.526 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 09:48:51 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z[106271]: 2023-10-08T09:48:51.309+0000 7fb2b1541200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:48:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:51.705 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:51 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:51.379+0000 7fd528ec5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:48:52.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 -- 172.21.15.97:0/188966534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 msgr2=0x7f74e0100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:52.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 --2- 172.21.15.97:0/188966534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e0100d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74d40099c0 tx=0x7f74d4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 -- 172.21.15.97:0/188966534 shutdown_connections 2023-10-08T09:48:52.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 --2- 172.21.15.97:0/188966534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 --2- 172.21.15.97:0/188966534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 0x7f74e00ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 --2- 172.21.15.97:0/188966534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e0100d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 -- 172.21.15.97:0/188966534 >> 172.21.15.97:0/188966534 conn(0x7f74e00facf0 msgr2=0x7f74e00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:52.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 -- 172.21.15.97:0/188966534 shutdown_connections 2023-10-08T09:48:52.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.064+0000 7f74e5fef700 1 -- 172.21.15.97:0/188966534 wait complete. 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 Processor -- start 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 -- start start 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 0x7f74e0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019e110 con 0x7f74e0100930 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019e280 con 0x7f74e00ff730 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74e5fef700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019e440 con 0x7f74e01012d0 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.065+0000 7f74deffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:52.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53982/0 (socket says 172.21.15.97:53982) 2023-10-08T09:48:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74df7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 0x7f74e0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 -- 172.21.15.97:0/4261207807 learned_addr learned my addr 172.21.15.97:0/4261207807 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74dffff700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 -- 172.21.15.97:0/4261207807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 msgr2=0x7f74e019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 -- 172.21.15.97:0/4261207807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 msgr2=0x7f74e0069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 0x7f74e0069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 -- 172.21.15.97:0/4261207807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d40095f0 con 0x7f74e0100930 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74dffff700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74deffd700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b580 tx=0x7f74d4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d401d070 con 0x7f74e0100930 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7f74d400bd10 con 0x7f74e0100930 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e019e6c0 con 0x7f74e0100930 2023-10-08T09:48:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4022c80 con 0x7f74e0100930 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.066+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e019ead0 con 0x7f74e0100930 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.067+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f74d402c440 con 0x7f74e0100930 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.067+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e0106a20 con 0x7f74e0100930 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.067+0000 7f74dcff9700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 0x7f74c803afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.067+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74d404c940 con 0x7f74e0100930 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.067+0000 7f74df7fe700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 0x7f74c803afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.068+0000 7f74df7fe700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 0x7f74c803afb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74e0100790 tx=0x7f74d0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:52.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.072+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f74d4016970 con 0x7f74e0100930 2023-10-08T09:48:52.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.143+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7f74d4022500 con 0x7f74e0100930 2023-10-08T09:48:52.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:52 smithi097 ceph-mon[99491]: Standby manager daemon z started 2023-10-08T09:48:52.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:52 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:48:52.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:52 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:52.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:52 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:48:52.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:52 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:52.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.281+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f74e004e4c0 con 0x7f74e0100930 2023-10-08T09:48:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.286+0000 7f74dcff9700 1 -- 172.21.15.97:0/4261207807 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) v1 ==== 106+0+0 (secure 0 0 0) 0x7f74d40270b0 con 0x7f74e0100930 2023-10-08T09:48:52.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 msgr2=0x7f74c803afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:52.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 0x7f74c803afb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74e0100790 tx=0x7f74d0007400 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 msgr2=0x7f74e019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b580 tx=0x7f74d4005b00 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 shutdown_connections 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74c8038af0 0x7f74c803afb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74e01012d0 0x7f74e019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00ff730 0x7f74e0069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 --2- 172.21.15.97:0/4261207807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74e0100930 0x7f74e019b380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.288+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 >> 172.21.15.97:0/4261207807 conn(0x7f74e00facf0 msgr2=0x7f74e0103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:52.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.289+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 shutdown_connections 2023-10-08T09:48:52.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:52.289+0000 7f74e5fef700 1 -- 172.21.15.97:0/4261207807 wait complete. 2023-10-08T09:48:52.404 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-mon[103156]: Standby manager daemon z started 2023-10-08T09:48:52.404 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:48:52.404 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:52.404 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:48:52.404 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:52.405 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:52.077+0000 7fd528ec5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:48:52.405 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:52.210+0000 7fd528ec5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:48:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:52 smithi186 ceph-mon[104393]: Standby manager daemon z started 2023-10-08T09:48:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:52 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:48:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:52 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:52 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:48:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:52 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.186:0/4032938520' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:52.705 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:52.403+0000 7fd528ec5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:48:52.705 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:52.516+0000 7fd528ec5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:48:53.144 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:52.734+0000 7fd528ec5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:48:53.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:53 smithi097 ceph-mon[99491]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:53.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:53 smithi097 ceph-mon[99491]: mgrmap e14: x(active, since 66s), standbys: z 2023-10-08T09:48:53.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:53 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:48:53.379 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:53 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4261207807' entity='client.admin' 2023-10-08T09:48:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:53 smithi178 ceph-mon[103156]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:53 smithi178 ceph-mon[103156]: mgrmap e14: x(active, since 66s), standbys: z 2023-10-08T09:48:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:53 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:48:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:53 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4261207807' entity='client.admin' 2023-10-08T09:48:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:53 smithi186 ceph-mon[104393]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:53 smithi186 ceph-mon[104393]: mgrmap e14: x(active, since 66s), standbys: z 2023-10-08T09:48:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:53 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:48:53.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:53 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4261207807' entity='client.admin' 2023-10-08T09:48:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 -- 172.21.15.97:0/552388186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 msgr2=0x7f5c880ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 --2- 172.21.15.97:0/552388186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c880ffdc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009990 tx=0x7f5c78009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 -- 172.21.15.97:0/552388186 shutdown_connections 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 --2- 172.21.15.97:0/552388186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 0x7f5c88107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 --2- 172.21.15.97:0/552388186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 --2- 172.21.15.97:0/552388186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c880ffdc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 -- 172.21.15.97:0/552388186 >> 172.21.15.97:0/552388186 conn(0x7f5c880fa450 msgr2=0x7f5c880fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 -- 172.21.15.97:0/552388186 shutdown_connections 2023-10-08T09:48:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.659+0000 7f5c90187700 1 -- 172.21.15.97:0/552388186 wait complete. 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 Processor -- start 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 -- start start 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 0x7f5c88101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88102a10 con 0x7f5c880ff940 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88102b80 con 0x7f5c88100300 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c90187700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881a05a0 con 0x7f5c88100ce0 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53994/0 (socket says 172.21.15.97:53994) 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 -- 172.21.15.97:0/3753907284 learned_addr learned my addr 172.21.15.97:0/3753907284 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8e724700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8d722700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 0x7f5c88101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 -- 172.21.15.97:0/3753907284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 msgr2=0x7f5c88102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 -- 172.21.15.97:0/3753907284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 msgr2=0x7f5c88101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 0x7f5c88101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 -- 172.21.15.97:0/3753907284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780095f0 con 0x7f5c880ff940 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8e724700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.660+0000 7f5c8df23700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d950 tx=0x7f5c8400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.661+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84009b40 con 0x7f5c880ff940 2023-10-08T09:48:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.661+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c881a0830 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.661+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7f5c84009ca0 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.661+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400f3c0 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.661+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c881a0d30 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.662+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88107530 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.662+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7f5c84010460 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.662+0000 7f5c7effd700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 0x7f5c7406f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.663+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c8408b1b0 con 0x7f5c880ff940 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.663+0000 7f5c8d722700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 0x7f5c7406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.663+0000 7f5c8d722700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 0x7f5c7406f430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800b5c0 tx=0x7f5c78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.667+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5c84056600 con 0x7f5c880ff940 2023-10-08T09:48:53.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.893+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f5c8810b630 con 0x7f5c880ff940 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.899+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5c84059c20 con 0x7f5c880ff940 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.900+0000 7f5c7effd700 1 -- 172.21.15.97:0/3753907284 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c8408aa40 con 0x7f5c880ff940 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 msgr2=0x7f5c7406f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 0x7f5c7406f430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800b5c0 tx=0x7f5c78005fb0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 msgr2=0x7f5c88103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d950 tx=0x7f5c8400dd10 comp rx=0 tx=0).stop 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 shutdown_connections 2023-10-08T09:48:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c7406cf70 0x7f5c7406f430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c88100ce0 0x7f5c88102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c880ff940 0x7f5c88103690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 --2- 172.21.15.97:0/3753907284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c88100300 0x7f5c88101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 >> 172.21.15.97:0/3753907284 conn(0x7f5c880fa450 msgr2=0x7f5c880fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 shutdown_connections 2023-10-08T09:48:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:53.902+0000 7f5c90187700 1 -- 172.21.15.97:0/3753907284 wait complete. 2023-10-08T09:48:53.955 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 09:48:53 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y[104373]: 2023-10-08T09:48:53.592+0000 7fd528ec5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:48:54.368 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi097') 2023-10-08T09:48:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 -- 172.21.15.97:0/2108347748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 msgr2=0x7fbdf8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/2108347748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf8102240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009990 tx=0x7fbde8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 -- 172.21.15.97:0/2108347748 shutdown_connections 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/2108347748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80ff240 0x7fbdf8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/2108347748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf8102240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/2108347748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf80fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 -- 172.21.15.97:0/2108347748 >> 172.21.15.97:0/2108347748 conn(0x7fbdf80fa480 msgr2=0x7fbdf80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 -- 172.21.15.97:0/2108347748 shutdown_connections 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 -- 172.21.15.97:0/2108347748 wait complete. 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.373+0000 7fbdfd52a700 1 Processor -- start 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 -- start start 2023-10-08T09:48:54.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 0x7fbdf8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819e1e0 con 0x7fbdf8101e20 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819e320 con 0x7fbdf80fe8a0 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819e4b0 con 0x7fbdf80ff240 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54010/0 (socket says 172.21.15.97:54010) 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 -- 172.21.15.97:0/1917876403 learned_addr learned my addr 172.21.15.97:0/1917876403 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf67fc700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 0x7fbdf8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf6ffd700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 -- 172.21.15.97:0/1917876403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 msgr2=0x7fbdf8196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 0x7fbdf8196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 -- 172.21.15.97:0/1917876403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 msgr2=0x7fbdf81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf81966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 -- 172.21.15.97:0/1917876403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80095f0 con 0x7fbdf8101e20 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf67fc700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 0x7fbdf8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf6ffd700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdf77fe700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000add0 tx=0x7fbdf000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000f070 con 0x7fbdf8101e20 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdf0007b30 con 0x7fbdf8101e20 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.374+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf819e790 con 0x7fbdf8101e20 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.375+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00052d0 con 0x7fbdf8101e20 2023-10-08T09:48:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.375+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf819ec40 con 0x7fbdf8101e20 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.375+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8107e90 con 0x7fbdf8101e20 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.377+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7fbdf000c940 con 0x7fbdf8101e20 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.377+0000 7fbddffff700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 0x7fbde006f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.377+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbdf008af10 con 0x7fbdf8101e20 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.377+0000 7fbdf6ffd700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 0x7fbde006f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.378+0000 7fbdf6ffd700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 0x7fbde006f430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbde800b5c0 tx=0x7fbde8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.381+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbdf00563e0 con 0x7fbdf8101e20 2023-10-08T09:48:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.422+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fbdf00567b0 con 0x7fbdf8101e20 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: Standby manager daemon y started 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:54.558 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3753907284' entity='client.admin' 2023-10-08T09:48:54.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:54.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:54.559 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.593+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7fbdf810c830 con 0x7fbdf8101e20 2023-10-08T09:48:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.597+0000 7fbddffff700 1 -- 172.21.15.97:0/1917876403 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v16)=0 v16) v1 ==== 116+0+0 (secure 0 0 0) 0x7fbdf0029ec0 con 0x7fbdf8101e20 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 msgr2=0x7fbde006f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 0x7fbde006f430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbde800b5c0 tx=0x7fbde8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 msgr2=0x7fbdf819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000add0 tx=0x7fbdf000c330 comp rx=0 tx=0).stop 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 shutdown_connections 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbde006cf70 0x7fbde006f430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8101e20 0x7fbdf819dc10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf80fe8a0 0x7fbdf81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 --2- 172.21.15.97:0/1917876403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdf80ff240 0x7fbdf8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 >> 172.21.15.97:0/1917876403 conn(0x7fbdf80fa480 msgr2=0x7fbdf8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 shutdown_connections 2023-10-08T09:48:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:54.599+0000 7fbdfd52a700 1 -- 172.21.15.97:0/1917876403 wait complete. 2023-10-08T09:48:54.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: Standby manager daemon y started 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3753907284' entity='client.admin' 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:54.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: Standby manager daemon y started 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:48:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.? 172.21.15.178:0/3977801343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3753907284' entity='client.admin' 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:48:54.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 -- 172.21.15.97:0/1246087880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 msgr2=0x7f3818101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 --2- 172.21.15.97:0/1246087880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f3818101ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3814009990 tx=0x7f3814009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 -- 172.21.15.97:0/1246087880 shutdown_connections 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 --2- 172.21.15.97:0/1246087880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 0x7f3818108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 --2- 172.21.15.97:0/1246087880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f38180fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 --2- 172.21.15.97:0/1246087880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f3818101ff0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 -- 172.21.15.97:0/1246087880 >> 172.21.15.97:0/1246087880 conn(0x7f38180fa450 msgr2=0x7f38180fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 -- 172.21.15.97:0/1246087880 shutdown_connections 2023-10-08T09:48:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.508+0000 7f38203a5700 1 -- 172.21.15.97:0/1246087880 wait complete. 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 Processor -- start 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 -- start start 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f38180ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 0x7f3818100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818100760 con 0x7f38180feb10 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181008d0 con 0x7f3818101b90 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f38203a5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181a0770 con 0x7f3818102530 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54020/0 (socket says 172.21.15.97:54020) 2023-10-08T09:48:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381d940700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f38180ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 -- 172.21.15.97:0/3364316323 learned_addr learned my addr 172.21.15.97:0/3364316323 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e942700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 0x7f3818100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 -- 172.21.15.97:0/3364316323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 msgr2=0x7f3818100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 0x7f3818100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 -- 172.21.15.97:0/3364316323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 msgr2=0x7f38180ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f38180ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 -- 172.21.15.97:0/3364316323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140095f0 con 0x7f38180feb10 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381d940700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f38180ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.509+0000 7f381e141700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f380c00e970 tx=0x7f380c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.510+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c009790 con 0x7f38180feb10 2023-10-08T09:48:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.510+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f380c00eec0 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.510+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c003c90 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.510+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38181a09d0 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.510+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38181a0ea0 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.511+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818107f10 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.512+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f380c004870 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.512+0000 7f380b7fe700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 0x7f38040a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.512+0000 7f381d940700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 0x7f38040a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.512+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f380c014070 con 0x7f38180feb10 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.513+0000 7f381d940700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 0x7f38040a3930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3814004360 tx=0x7f3814005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.516+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f380c096630 con 0x7f38180feb10 2023-10-08T09:48:55.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:55 smithi097 ceph-mon[99491]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-10-08T09:48:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:48:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:55 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1917876403' entity='client.admin' 2023-10-08T09:48:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:55 smithi178 ceph-mon[103156]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-10-08T09:48:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:48:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:55 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1917876403' entity='client.admin' 2023-10-08T09:48:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.716+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f381810c870 con 0x7f38180feb10 2023-10-08T09:48:55.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.720+0000 7f380b7fe700 1 -- 172.21.15.97:0/3364316323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) v1 ==== 120+0+0 (secure 0 0 0) 0x7f380c099c50 con 0x7f38180feb10 2023-10-08T09:48:55.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.722+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 msgr2=0x7f38040a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:55.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.722+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 0x7f38040a3930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3814004360 tx=0x7f3814005fd0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.722+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 msgr2=0x7f3818101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.722+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f380c00e970 tx=0x7f380c00ec80 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 shutdown_connections 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38040a1470 0x7f38040a3930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3818102530 0x7f3818100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38180feb10 0x7f3818101450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 --2- 172.21.15.97:0/3364316323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3818101b90 0x7f38180ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 >> 172.21.15.97:0/3364316323 conn(0x7f38180fa450 msgr2=0x7f3818105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 shutdown_connections 2023-10-08T09:48:55.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.723+0000 7f38203a5700 1 -- 172.21.15.97:0/3364316323 wait complete. 2023-10-08T09:48:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:55 smithi186 ceph-mon[104393]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-10-08T09:48:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:48:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:55 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1917876403' entity='client.admin' 2023-10-08T09:48:56.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3154059365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78100840 msgr2=0x7fbb78100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3154059365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78100840 0x7fbb78100c60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0099c0 tx=0x7fbb6c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3154059365 shutdown_connections 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3154059365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb781023e0 0x7fbb78108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3154059365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78100840 0x7fbb78100c60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3154059365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb78101a40 0x7fbb78101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3154059365 >> 172.21.15.97:0/3154059365 conn(0x7fbb780fbde0 msgr2=0x7fbb780fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:56.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3154059365 shutdown_connections 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.814+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3154059365 wait complete. 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 Processor -- start 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 -- start start 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb78100840 0x7fbb78196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb781023e0 0x7fbb7819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819e180 con 0x7fbb78101a40 2023-10-08T09:48:56.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819e2c0 con 0x7fbb781023e0 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7ea6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78072a20 con 0x7fbb78100840 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7c80a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb78100840 0x7fbb78196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54038/0 (socket says 172.21.15.97:54038) 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb77fff700 1 -- 172.21.15.97:0/3486228720 learned_addr learned my addr 172.21.15.97:0/3486228720 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.815+0000 7fbb7d00b700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb781023e0 0x7fbb7819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 -- 172.21.15.97:0/3486228720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb78100840 msgr2=0x7fbb78196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb78100840 0x7fbb78196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 -- 172.21.15.97:0/3486228720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb781023e0 msgr2=0x7fbb7819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb781023e0 0x7fbb7819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 -- 172.21.15.97:0/3486228720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0095f0 con 0x7fbb78101a40 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb77fff700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400e9d0 tx=0x7fbb6400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.816+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400cbd0 con 0x7fbb78101a40 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.817+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb6400ee70 con 0x7fbb78101a40 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.817+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64018680 con 0x7fbb78101a40 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.817+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb78072d00 con 0x7fbb78101a40 2023-10-08T09:48:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.818+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb78073180 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.819+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7804e4c0 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.819+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fbb64010430 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.820+0000 7fbb75ffb700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 0x7fbb680a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.820+0000 7fbb7c80a700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 0x7fbb680a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.820+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb64014070 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.820+0000 7fbb7c80a700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 0x7fbb680a3a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0036c0 tx=0x7fbb6c005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:55.824+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb640966c0 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.024+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fbb7810bf00 con 0x7fbb78101a40 2023-10-08T09:48:56.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.027+0000 7fbb75ffb700 1 -- 172.21.15.97:0/3486228720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v18)=0 v18) v1 ==== 108+0+0 (secure 0 0 0) 0x7fbb64099ce0 con 0x7fbb78101a40 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.029+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 msgr2=0x7fbb680a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.029+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 0x7fbb680a3a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0036c0 tx=0x7fbb6c005ad0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.029+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 msgr2=0x7fbb78196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.029+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400e9d0 tx=0x7fbb6400ece0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 shutdown_connections 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbb680a1540 0x7fbb680a3a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb781023e0 0x7fbb7819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb78100840 0x7fbb78196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 --2- 172.21.15.97:0/3486228720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb78101a40 0x7fbb78196bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 >> 172.21.15.97:0/3486228720 conn(0x7fbb780fbde0 msgr2=0x7fbb78104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 shutdown_connections 2023-10-08T09:48:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:56.030+0000 7fbb7ea6e700 1 -- 172.21.15.97:0/3486228720 wait complete. 2023-10-08T09:48:57.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3364316323' entity='client.admin' 2023-10-08T09:48:57.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:56 smithi097 ceph-mon[99491]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:57.022 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3486228720' entity='client.admin' 2023-10-08T09:48:57.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3364316323' entity='client.admin' 2023-10-08T09:48:57.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:56 smithi186 ceph-mon[104393]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:57.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3486228720' entity='client.admin' 2023-10-08T09:48:57.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3364316323' entity='client.admin' 2023-10-08T09:48:57.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:56 smithi178 ceph-mon[103156]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:57.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3486228720' entity='client.admin' 2023-10-08T09:48:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- 172.21.15.97:0/2559524495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 msgr2=0x7fce24106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2559524495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24106060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce10009990 tx=0x7fce10009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- 172.21.15.97:0/2559524495 shutdown_connections 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2559524495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 0x7fce241089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2559524495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 0x7fce24103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2559524495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24106060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- 172.21.15.97:0/2559524495 >> 172.21.15.97:0/2559524495 conn(0x7fce240fad10 msgr2=0x7fce240fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- 172.21.15.97:0/2559524495 shutdown_connections 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- 172.21.15.97:0/2559524495 wait complete. 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 Processor -- start 2023-10-08T09:48:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.413+0000 7fce2aceb700 1 -- start start 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 0x7fce24196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 0x7fce2419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e290 con 0x7fce24103c70 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e3d0 con 0x7fce241065a0 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e560 con 0x7fce24101340 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54050/0 (socket says 172.21.15.97:54050) 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 -- 172.21.15.97:0/2479273085 learned_addr learned my addr 172.21.15.97:0/2479273085 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce29288700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 0x7fce2419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce28a87700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 0x7fce24196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 -- 172.21.15.97:0/2479273085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 msgr2=0x7fce24196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 0x7fce24196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 -- 172.21.15.97:0/2479273085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 msgr2=0x7fce2419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 0x7fce2419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 -- 172.21.15.97:0/2479273085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100095f0 con 0x7fce24103c70 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce23fff700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce100047a0 tx=0x7fce10003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1001d070 con 0x7fce24103c70 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce10022470 con 0x7fce24103c70 2023-10-08T09:48:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2419e7e0 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.414+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000f6c0 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.415+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2419ebf0 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.416+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce240fc890 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.417+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fce100228e0 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.417+0000 7fce21ffb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 0x7fce140a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.418+0000 7fce28a87700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 0x7fce140a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.418+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fce100cc540 con 0x7fce24103c70 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.418+0000 7fce28a87700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 0x7fce140a38b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce18006fd0 tx=0x7fce18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.422+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fce10097900 con 0x7fce24103c70 2023-10-08T09:48:57.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.623+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7fce2404e4c0 con 0x7fce24103c70 2023-10-08T09:48:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.626+0000 7fce21ffb700 1 -- 172.21.15.97:0/2479273085 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v19)=0 v19) v1 ==== 116+0+0 (secure 0 0 0) 0x7fce1009af20 con 0x7fce24103c70 2023-10-08T09:48:57.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 msgr2=0x7fce140a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 0x7fce140a38b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce18006fd0 tx=0x7fce18008040 comp rx=0 tx=0).stop 2023-10-08T09:48:57.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 msgr2=0x7fce24196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce100047a0 tx=0x7fce10003800 comp rx=0 tx=0).stop 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 shutdown_connections 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fce140a13f0 0x7fce140a38b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce241065a0 0x7fce2419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce24101340 0x7fce24196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 --2- 172.21.15.97:0/2479273085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce24103c70 0x7fce24196c90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 >> 172.21.15.97:0/2479273085 conn(0x7fce240fad10 msgr2=0x7fce24101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 shutdown_connections 2023-10-08T09:48:57.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.629+0000 7fce2aceb700 1 -- 172.21.15.97:0/2479273085 wait complete. 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- 172.21.15.97:0/740688905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84241012c0 msgr2=0x7f8424107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 --2- 172.21.15.97:0/740688905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84241012c0 0x7f8424107b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f841c00b0d0 tx=0x7f841c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- 172.21.15.97:0/740688905 shutdown_connections 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 --2- 172.21.15.97:0/740688905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84241012c0 0x7f8424107b70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 --2- 172.21.15.97:0/740688905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84240ff720 0x7f84240ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 --2- 172.21.15.97:0/740688905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8424100920 0x7f8424100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- 172.21.15.97:0/740688905 >> 172.21.15.97:0/740688905 conn(0x7f84240face0 msgr2=0x7f84240fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- 172.21.15.97:0/740688905 shutdown_connections 2023-10-08T09:48:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- 172.21.15.97:0/740688905 wait complete. 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 Processor -- start 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.455+0000 7f842b54b700 1 -- start start 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84240ff720 0x7f84241955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84241012c0 0x7f842419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419d130 con 0x7f8424100920 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419d270 con 0x7f84240ff720 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419d400 con 0x7f84241012c0 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54064/0 (socket says 172.21.15.97:54064) 2023-10-08T09:48:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 -- 172.21.15.97:0/4146328681 learned_addr learned my addr 172.21.15.97:0/4146328681 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f84292e7700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84240ff720 0x7f84241955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8429ae8700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84241012c0 0x7f842419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 -- 172.21.15.97:0/4146328681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84241012c0 msgr2=0x7f842419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84241012c0 0x7f842419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 -- 172.21.15.97:0/4146328681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84240ff720 msgr2=0x7f84241955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84240ff720 0x7f84241955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 -- 172.21.15.97:0/4146328681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009d00 con 0x7f8424100920 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f8428ae6700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f84140099a0 tx=0x7f8414009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414012070 con 0x7f8424100920 2023-10-08T09:48:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f841400d8f0 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400c910 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8424068e80 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.456+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8424069380 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.457+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842410b720 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.458+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f8414007be0 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.459+0000 7f841a7fc700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 0x7f84100a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.459+0000 7f84292e7700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 0x7f84100a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.459+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f84140cbf00 con 0x7f8424100920 2023-10-08T09:48:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.460+0000 7f84292e7700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 0x7f84100a3930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8424100780 tx=0x7f842000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.463+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8414097230 con 0x7f8424100920 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.669+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f842404e4c0 con 0x7f8424100920 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.674+0000 7f841a7fc700 1 -- 172.21.15.97:0/4146328681 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v20)=0 v20) v1 ==== 119+0+0 (secure 0 0 0) 0x7f841409a850 con 0x7f8424100920 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 msgr2=0x7f84100a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 0x7f84100a3930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8424100780 tx=0x7f842000b410 comp rx=0 tx=0).stop 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 msgr2=0x7f8424195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f84140099a0 tx=0x7f8414009cb0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 shutdown_connections 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f84100a1470 0x7f84100a3930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84241012c0 0x7f842419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84240ff720 0x7f84241955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 --2- 172.21.15.97:0/4146328681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8424100920 0x7f8424195b30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 >> 172.21.15.97:0/4146328681 conn(0x7f84240face0 msgr2=0x7f8424103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 shutdown_connections 2023-10-08T09:48:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.676+0000 7f842b54b700 1 -- 172.21.15.97:0/4146328681 wait complete. 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 -- 172.21.15.97:0/4059885876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 msgr2=0x7f8a9c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/4059885876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c100080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009990 tx=0x7f8a98009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 -- 172.21.15.97:0/4059885876 shutdown_connections 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/4059885876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 0x7f8a9c107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/4059885876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c100080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/4059885876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 -- 172.21.15.97:0/4059885876 >> 172.21.15.97:0/4059885876 conn(0x7f8a9c0fa480 msgr2=0x7f8a9c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 -- 172.21.15.97:0/4059885876 shutdown_connections 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.713+0000 7f8aa3cea700 1 -- 172.21.15.97:0/4059885876 wait complete. 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 Processor -- start 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 -- start start 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 0x7f8a9c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c19e280 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c19e3c0 con 0x7f8a9c101340 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa3cea700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c19e550 con 0x7f8a9c100980 2023-10-08T09:48:57.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa1a86700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa1a86700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54076/0 (socket says 172.21.15.97:54076) 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa1a86700 1 -- 172.21.15.97:0/1805880872 learned_addr learned my addr 172.21.15.97:0/1805880872 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa2287700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 0x7f8a9c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.714+0000 7f8aa1285700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 -- 172.21.15.97:0/1805880872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 msgr2=0x7f8a9c196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 -- 172.21.15.97:0/1805880872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 msgr2=0x7f8a9c19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 0x7f8a9c19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 -- 172.21.15.97:0/1805880872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a980095f0 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1285700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:48:57.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa1a86700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000c00 tx=0x7f8a98005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800bb70 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a98004cd0 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c108500 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98005600 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.715+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c1089a0 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.716+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c106d00 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.717+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f8a98025070 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.718+0000 7f8a92ffd700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 0x7f8a880a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.718+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a980d0770 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.718+0000 7f8aa1285700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 0x7f8a880a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.718+0000 7f8aa1285700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 0x7f8a880a38b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0ff990 tx=0x7f8a8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:57.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.723+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8a9809bb30 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.932+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f8a9c04e4c0 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.936+0000 7f8a92ffd700 1 -- 172.21.15.97:0/1805880872 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v21)=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8a9809f150 con 0x7f8a9c0ffc60 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 msgr2=0x7f8a880a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 0x7f8a880a38b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0ff990 tx=0x7f8a8c009260 comp rx=0 tx=0).stop 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 msgr2=0x7f8a9c196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000c00 tx=0x7f8a98005ef0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 shutdown_connections 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8a880a13f0 0x7f8a880a38b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a9c101340 0x7f8a9c19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a9c0ffc60 0x7f8a9c196790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 --2- 172.21.15.97:0/1805880872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a9c100980 0x7f8a9c196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:57.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 >> 172.21.15.97:0/1805880872 conn(0x7f8a9c0fa480 msgr2=0x7f8a9c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:57.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 shutdown_connections 2023-10-08T09:48:57.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:57.938+0000 7f8aa3cea700 1 -- 172.21.15.97:0/1805880872 wait complete. 2023-10-08T09:48:58.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2479273085' entity='client.admin' 2023-10-08T09:48:58.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4146328681' entity='client.admin' 2023-10-08T09:48:58.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:58 smithi097 ceph-mon[99491]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:58.829 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:48:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1805880872' entity='client.admin' 2023-10-08T09:48:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2479273085' entity='client.admin' 2023-10-08T09:48:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4146328681' entity='client.admin' 2023-10-08T09:48:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:58 smithi178 ceph-mon[103156]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:58.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:48:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1805880872' entity='client.admin' 2023-10-08T09:48:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2479273085' entity='client.admin' 2023-10-08T09:48:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4146328681' entity='client.admin' 2023-10-08T09:48:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:58 smithi186 ceph-mon[104393]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:48:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:48:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1805880872' entity='client.admin' 2023-10-08T09:48:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:48:59.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 -- 172.21.15.97:0/1260769206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea41043f0 msgr2=0x7f6ea4104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/1260769206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea41043f0 0x7f6ea4104870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c01c0d0 tx=0x7f6e9c01c3e0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 -- 172.21.15.97:0/1260769206 shutdown_connections 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/1260769206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea41043f0 0x7f6ea4104870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/1260769206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea40ff0b0 0x7f6ea41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/1260769206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4101a20 0x7f6ea4103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.616+0000 7f6ea9f68700 1 -- 172.21.15.97:0/1260769206 >> 172.21.15.97:0/1260769206 conn(0x7f6ea40facf0 msgr2=0x7f6ea40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.617+0000 7f6ea9f68700 1 -- 172.21.15.97:0/1260769206 shutdown_connections 2023-10-08T09:48:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.617+0000 7f6ea9f68700 1 -- 172.21.15.97:0/1260769206 wait complete. 2023-10-08T09:48:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 Processor -- start 2023-10-08T09:48:59.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 -- start start 2023-10-08T09:48:59.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea40ff0b0 0x7f6ea4194580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4199ae0 0x7f6ea419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419c4a0 con 0x7f6ea40ff0b0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419c5e0 con 0x7f6ea4199ae0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.618+0000 7f6ea9f68700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea419c770 con 0x7f6ea41043f0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4199ae0 0x7f6ea419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:55806/0 (socket says 172.21.15.97:55806) 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 -- 172.21.15.97:0/3440259310 learned_addr learned my addr 172.21.15.97:0/3440259310 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea37fe700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea40ff0b0 0x7f6ea4194580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 -- 172.21.15.97:0/3440259310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4199ae0 msgr2=0x7f6ea419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4199ae0 0x7f6ea419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 -- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea40ff0b0 msgr2=0x7f6ea4194580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea40ff0b0 0x7f6ea4194580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.619+0000 7f6ea2ffd700 1 -- 172.21.15.97:0/3440259310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c01ad10 con 0x7f6ea41043f0 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.620+0000 7f6ea2ffd700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400b530 tx=0x7f6e9400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.621+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94011830 con 0x7f6ea41043f0 2023-10-08T09:48:59.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.621+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e94011e30 con 0x7f6ea41043f0 2023-10-08T09:48:59.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.621+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94010db0 con 0x7f6ea41043f0 2023-10-08T09:48:59.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.621+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea419ca50 con 0x7f6ea41043f0 2023-10-08T09:48:59.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.621+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea419cf00 con 0x7f6ea41043f0 2023-10-08T09:48:59.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.622+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e90005110 con 0x7f6ea41043f0 2023-10-08T09:48:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.623+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f6e94011990 con 0x7f6ea41043f0 2023-10-08T09:48:59.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.624+0000 7f6ea0ff9700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 0x7f6e8c0a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.624+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e940ca7f0 con 0x7f6ea41043f0 2023-10-08T09:48:59.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.629+0000 7f6ea37fe700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 0x7f6e8c0a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.630+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e94095ac0 con 0x7f6ea41043f0 2023-10-08T09:48:59.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.643+0000 7f6ea37fe700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 0x7f6e8c0a3a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e98007c60 tx=0x7f6e980073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:59.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- 172.21.15.97:0/789695087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0101ae0 msgr2=0x7f07e0101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 --2- 172.21.15.97:0/789695087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0101ae0 0x7f07e0101f40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009990 tx=0x7f07dc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- 172.21.15.97:0/789695087 shutdown_connections 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 --2- 172.21.15.97:0/789695087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0102480 0x7f07e0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 --2- 172.21.15.97:0/789695087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e00feb00 0x7f07e00fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 --2- 172.21.15.97:0/789695087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e0101ae0 0x7f07e0101f40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- 172.21.15.97:0/789695087 >> 172.21.15.97:0/789695087 conn(0x7f07e00fa480 msgr2=0x7f07e00fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- 172.21.15.97:0/789695087 shutdown_connections 2023-10-08T09:48:59.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- 172.21.15.97:0/789695087 wait complete. 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 Processor -- start 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.467+0000 7f07e7725700 1 -- start start 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101ae0 0x7f07e00ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 0x7f07e00fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0100770 con 0x7f07e00feb00 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01008e0 con 0x7f07e0102480 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01a0650 con 0x7f07e0101ae0 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54090/0 (socket says 172.21.15.97:54090) 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 -- 172.21.15.97:0/364174500 learned_addr learned my addr 172.21.15.97:0/364174500 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e4cc0700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101ae0 0x7f07e00ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e5cc2700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 0x7f07e00fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 -- 172.21.15.97:0/364174500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101ae0 msgr2=0x7f07e00ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101ae0 0x7f07e00ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 -- 172.21.15.97:0/364174500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 msgr2=0x7f07e00fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 0x7f07e00fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 -- 172.21.15.97:0/364174500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0095f0 con 0x7f07e00feb00 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e5cc2700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 0x7f07e00fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e54c1700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d980 tx=0x7f07d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4004020 con 0x7f07e00feb00 2023-10-08T09:48:59.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f07d4009730 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003a10 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.468+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e01a08e0 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.469+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01a0de0 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.470+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0107d90 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.471+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f07d4009b80 con 0x7f07e00feb00 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.471+0000 7f07d27fc700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 0x7f07cc0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:48:59.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.472+0000 7f07e4cc0700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 0x7f07cc0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.472+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07d40cac40 con 0x7f07e00feb00 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.472+0000 7f07e4cc0700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 0x7f07cc0a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b5c0 tx=0x7f07dc005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.476+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f07d40d1080 con 0x7f07e00feb00 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.730+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7f07e010c0b0 con 0x7f07e00feb00 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.734+0000 7f07d27fc700 1 -- 172.21.15.97:0/364174500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07d4095f70 con 0x7f07e00feb00 2023-10-08T09:48:59.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 msgr2=0x7f07cc0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 0x7f07cc0a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b5c0 tx=0x7f07dc005c00 comp rx=0 tx=0).stop 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 msgr2=0x7f07e01013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d980 tx=0x7f07d400dc90 comp rx=0 tx=0).stop 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 shutdown_connections 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f07cc0a1470 0x7f07cc0a3930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e0102480 0x7f07e00fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e00feb00 0x7f07e01013f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 --2- 172.21.15.97:0/364174500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07e0101ae0 0x7f07e00ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 >> 172.21.15.97:0/364174500 conn(0x7f07e00fa480 msgr2=0x7f07e0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 shutdown_connections 2023-10-08T09:48:59.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.737+0000 7f07e7725700 1 -- 172.21.15.97:0/364174500 wait complete. 2023-10-08T09:48:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.868+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f6e90005940 con 0x7f6ea41043f0 2023-10-08T09:48:59.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.875+0000 7f6ea0ff9700 1 -- 172.21.15.97:0/3440259310 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6e940150c0 con 0x7f6ea41043f0 2023-10-08T09:48:59.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 msgr2=0x7f6e8c0a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 0x7f6e8c0a3a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e98007c60 tx=0x7f6e980073d0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 msgr2=0x7f6ea4194ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400b530 tx=0x7f6e9400b840 comp rx=0 tx=0).stop 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 shutdown_connections 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e8c0a1540 0x7f6e8c0a3a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ea4199ae0 0x7f6ea419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ea40ff0b0 0x7f6ea4194580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 --2- 172.21.15.97:0/3440259310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ea41043f0 0x7f6ea4194ac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 >> 172.21.15.97:0/3440259310 conn(0x7f6ea40facf0 msgr2=0x7f6ea40fb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 shutdown_connections 2023-10-08T09:48:59.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.877+0000 7f6ea9f68700 1 -- 172.21.15.97:0/3440259310 wait complete. 2023-10-08T09:49:00.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 -- 172.21.15.97:0/3659068463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 msgr2=0x7ff100107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 --2- 172.21.15.97:0/3659068463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff100107b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800b0d0 tx=0x7ff0f800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 -- 172.21.15.97:0/3659068463 shutdown_connections 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 --2- 172.21.15.97:0/3659068463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff100107b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 --2- 172.21.15.97:0/3659068463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1000ff730 0x7ff1000ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 --2- 172.21.15.97:0/3659068463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100930 0x7ff100100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 -- 172.21.15.97:0/3659068463 >> 172.21.15.97:0/3659068463 conn(0x7ff1000facf0 msgr2=0x7ff1000fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.347+0000 7ff105645700 1 -- 172.21.15.97:0/3659068463 shutdown_connections 2023-10-08T09:49:00.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 -- 172.21.15.97:0/3659068463 wait complete. 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 Processor -- start 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 -- start start 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1000ff730 0x7ff100198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff100100930 0x7ff100198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001a0490 con 0x7ff1001012d0 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001a05d0 con 0x7ff1000ff730 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.348+0000 7ff105645700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001a0760 con 0x7ff100100930 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54136/0 (socket says 172.21.15.97:54136) 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 -- 172.21.15.97:0/1400125095 learned_addr learned my addr 172.21.15.97:0/1400125095 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0feffd700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1000ff730 0x7ff100198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0fe7fc700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff100100930 0x7ff100198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 -- 172.21.15.97:0/1400125095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff100100930 msgr2=0x7ff100198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff100100930 0x7ff100198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 -- 172.21.15.97:0/1400125095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1000ff730 msgr2=0x7ff100198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1000ff730 0x7ff100198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 -- 172.21.15.97:0/1400125095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8009d00 con 0x7ff1001012d0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0ff7fe700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d010 tx=0x7ff0f8009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800e040 con 0x7ff1001012d0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0f8012da0 con 0x7ff1001012d0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1001a09e0 con 0x7ff1001012d0 2023-10-08T09:49:00.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801de60 con 0x7ff1001012d0 2023-10-08T09:49:00.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.349+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1001a0ea0 con 0x7ff1001012d0 2023-10-08T09:49:00.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.350+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10004e4c0 con 0x7ff1001012d0 2023-10-08T09:49:00.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.351+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff0f8012900 con 0x7ff1001012d0 2023-10-08T09:49:00.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.352+0000 7ff0e7fff700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 0x7ff0e80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.352+0000 7ff0feffd700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 0x7ff0e80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.352+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff0f80ccec0 con 0x7ff1001012d0 2023-10-08T09:49:00.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.352+0000 7ff0feffd700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 0x7ff0e80a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005fd0 tx=0x7ff0f0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:00.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.356+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0f80981e0 con 0x7ff1001012d0 2023-10-08T09:49:00.382 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:49:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 -- 172.21.15.97:0/2530328336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330103cb0 msgr2=0x7f53301060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 --2- 172.21.15.97:0/2530328336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330103cb0 0x7f53301060a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5324009990 tx=0x7f5324009ca0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 -- 172.21.15.97:0/2530328336 shutdown_connections 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 --2- 172.21.15.97:0/2530328336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301065e0 0x7f53301089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 --2- 172.21.15.97:0/2530328336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330101380 0x7f5330103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 --2- 172.21.15.97:0/2530328336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330103cb0 0x7f53301060a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 -- 172.21.15.97:0/2530328336 >> 172.21.15.97:0/2530328336 conn(0x7f53300facf0 msgr2=0x7f53300fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 -- 172.21.15.97:0/2530328336 shutdown_connections 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.680+0000 7f533615b700 1 -- 172.21.15.97:0/2530328336 wait complete. 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 Processor -- start 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 -- start start 2023-10-08T09:49:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330103cb0 0x7f5330196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301065e0 0x7f533019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e1a0 con 0x7f5330101380 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e2e0 con 0x7f53301065e0 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f533615b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533019e470 con 0x7f5330103cb0 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f532f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f532f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54118/0 (socket says 172.21.15.97:54118) 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f532f7fe700 1 -- 172.21.15.97:0/2729116184 learned_addr learned my addr 172.21.15.97:0/2729116184 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f532f7fe700 1 -- 172.21.15.97:0/2729116184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330103cb0 msgr2=0x7f5330196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.681+0000 7f532f7fe700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330103cb0 0x7f5330196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532f7fe700 1 -- 172.21.15.97:0/2729116184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301065e0 msgr2=0x7f533019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532f7fe700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301065e0 0x7f533019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532f7fe700 1 -- 172.21.15.97:0/2729116184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240095f0 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532f7fe700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320009ac0 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533019e750 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f532000b6e0 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000f3c0 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.682+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533019eca0 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.683+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53300fc870 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.684+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5320010460 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.685+0000 7f532cff9700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 0x7f53180a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.685+0000 7f532effd700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 0x7f53180a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.685+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f53200cae80 con 0x7f5330101380 2023-10-08T09:49:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.685+0000 7f532effd700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 0x7f53180a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5324009990 tx=0x7f5324005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.689+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f53200961b0 con 0x7f5330101380 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.914+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f533004e4c0 con 0x7f5330101380 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.920+0000 7f532cff9700 1 -- 172.21.15.97:0/2729116184 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v24)=0 v24) v1 ==== 127+0+0 (secure 0 0 0) 0x7f53200160c0 con 0x7f5330101380 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 msgr2=0x7f53180a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 0x7f53180a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5324009990 tx=0x7f5324005fd0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 msgr2=0x7f5330196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dd40 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 shutdown_connections 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f53180a13f0 0x7f53180a38b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301065e0 0x7f533019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5330101380 0x7f5330196660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 --2- 172.21.15.97:0/2729116184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5330103cb0 0x7f5330196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 >> 172.21.15.97:0/2729116184 conn(0x7f53300facf0 msgr2=0x7f5330101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:00.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 shutdown_connections 2023-10-08T09:49:00.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:48:59.923+0000 7f533615b700 1 -- 172.21.15.97:0/2729116184 wait complete. 2023-10-08T09:49:00.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.556+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7ff10010ae30 con 0x7ff1001012d0 2023-10-08T09:49:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.563+0000 7ff0e7fff700 1 -- 172.21.15.97:0/1400125095 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff0f809b800 con 0x7ff1001012d0 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 msgr2=0x7ff0e80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 0x7ff0e80a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005fd0 tx=0x7ff0f0005e50 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 msgr2=0x7ff10019fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d010 tx=0x7ff0f8009430 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 shutdown_connections 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e80a14f0 0x7ff0e80a39b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1001012d0 0x7ff10019fec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1000ff730 0x7ff100198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 --2- 172.21.15.97:0/1400125095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff100100930 0x7ff100198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 >> 172.21.15.97:0/1400125095 conn(0x7ff1000facf0 msgr2=0x7ff1000fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:00.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 shutdown_connections 2023-10-08T09:49:00.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:00.566+0000 7ff105645700 1 -- 172.21.15.97:0/1400125095 wait complete. 2023-10-08T09:49:00.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/364174500' entity='client.admin' 2023-10-08T09:49:00.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:00 smithi097 ceph-mon[99491]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:00.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:00 smithi097 ceph-mon[99491]: from='client.? ' entity='client.admin' 2023-10-08T09:49:00.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2729116184' entity='client.admin' 2023-10-08T09:49:00.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1400125095' entity='client.admin' 2023-10-08T09:49:01.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/364174500' entity='client.admin' 2023-10-08T09:49:01.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:00 smithi186 ceph-mon[104393]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:01.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:00 smithi186 ceph-mon[104393]: from='client.? ' entity='client.admin' 2023-10-08T09:49:01.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2729116184' entity='client.admin' 2023-10-08T09:49:01.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1400125095' entity='client.admin' 2023-10-08T09:49:01.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/364174500' entity='client.admin' 2023-10-08T09:49:01.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:00 smithi178 ceph-mon[103156]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:01.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:00 smithi178 ceph-mon[103156]: from='client.? ' entity='client.admin' 2023-10-08T09:49:01.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2729116184' entity='client.admin' 2023-10-08T09:49:01.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1400125095' entity='client.admin' 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- 172.21.15.97:0/915362071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59901012d0 msgr2=0x7f5990107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 --2- 172.21.15.97:0/915362071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59901012d0 0x7f5990107b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f598800b0d0 tx=0x7f598800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- 172.21.15.97:0/915362071 shutdown_connections 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 --2- 172.21.15.97:0/915362071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59901012d0 0x7f5990107b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 --2- 172.21.15.97:0/915362071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59900ff730 0x7f59900ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 --2- 172.21.15.97:0/915362071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f5990100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- 172.21.15.97:0/915362071 >> 172.21.15.97:0/915362071 conn(0x7f59900facf0 msgr2=0x7f59900fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- 172.21.15.97:0/915362071 shutdown_connections 2023-10-08T09:49:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- 172.21.15.97:0/915362071 wait complete. 2023-10-08T09:49:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 Processor -- start 2023-10-08T09:49:01.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.507+0000 7f59981b2700 1 -- start start 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f599019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59901012d0 0x7f599019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599019e200 con 0x7f59900ff730 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599019e370 con 0x7f5990100930 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599019e530 con 0x7f59901012d0 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54146/0 (socket says 172.21.15.97:54146) 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 -- 172.21.15.97:0/154312241 learned_addr learned my addr 172.21.15.97:0/154312241 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:01.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f599574d700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f599019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 -- 172.21.15.97:0/154312241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59901012d0 msgr2=0x7f599019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59901012d0 0x7f599019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 -- 172.21.15.97:0/154312241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 msgr2=0x7f599019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f599019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 -- 172.21.15.97:0/154312241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988009d00 con 0x7f59900ff730 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f599574d700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f599019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5995f4e700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f598000c970 tx=0x7f598000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980004220 con 0x7f59900ff730 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5980007d40 con 0x7f59900ff730 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599019e810 con 0x7f59900ff730 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980003c00 con 0x7f59900ff730 2023-10-08T09:49:01.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.508+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599019ed60 con 0x7f59900ff730 2023-10-08T09:49:01.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.509+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5990106b40 con 0x7f59900ff730 2023-10-08T09:49:01.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.510+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5980004380 con 0x7f59900ff730 2023-10-08T09:49:01.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.511+0000 7f5986ffd700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 0x7f597c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.511+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f59800cabd0 con 0x7f59900ff730 2023-10-08T09:49:01.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.511+0000 7f599574d700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 0x7f597c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.512+0000 7f599574d700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 0x7f597c0a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f598c005950 tx=0x7f598c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.518+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5980095f00 con 0x7f59900ff730 2023-10-08T09:49:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.736+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f599004e4c0 con 0x7f59900ff730 2023-10-08T09:49:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.740+0000 7f5986ffd700 1 -- 172.21.15.97:0/154312241 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v26)=0 v26) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5980099520 con 0x7f59900ff730 2023-10-08T09:49:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 msgr2=0x7f597c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 0x7f597c0a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f598c005950 tx=0x7f598c0058e0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 msgr2=0x7f5990068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f598000c970 tx=0x7f598000cd30 comp rx=0 tx=0).stop 2023-10-08T09:49:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 shutdown_connections 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f597c0a1470 0x7f597c0a3930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59901012d0 0x7f599019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59900ff730 0x7f5990068cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 --2- 172.21.15.97:0/154312241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5990100930 0x7f599019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 >> 172.21.15.97:0/154312241 conn(0x7f59900facf0 msgr2=0x7f5990103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 shutdown_connections 2023-10-08T09:49:01.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.742+0000 7f59981b2700 1 -- 172.21.15.97:0/154312241 wait complete. 2023-10-08T09:49:01.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 -- 172.21.15.97:0/2778480870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 msgr2=0x7fdb7c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 --2- 172.21.15.97:0/2778480870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c107b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b0d0 tx=0x7fdb7400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 -- 172.21.15.97:0/2778480870 shutdown_connections 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 --2- 172.21.15.97:0/2778480870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c107b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 --2- 172.21.15.97:0/2778480870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c0ff730 0x7fdb7c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 --2- 172.21.15.97:0/2778480870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c100930 0x7fdb7c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 -- 172.21.15.97:0/2778480870 >> 172.21.15.97:0/2778480870 conn(0x7fdb7c0facf0 msgr2=0x7fdb7c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 -- 172.21.15.97:0/2778480870 shutdown_connections 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.782+0000 7fdb83b07700 1 -- 172.21.15.97:0/2778480870 wait complete. 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 Processor -- start 2023-10-08T09:49:01.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 -- start start 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c0ff730 0x7fdb7c068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c100930 0x7fdb7c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1a0460 con 0x7fdb7c1012d0 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1a05d0 con 0x7fdb7c100930 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb83b07700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1a0790 con 0x7fdb7c0ff730 2023-10-08T09:49:01.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb820a4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb820a4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54164/0 (socket says 172.21.15.97:54164) 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb820a4700 1 -- 172.21.15.97:0/978080663 learned_addr learned my addr 172.21.15.97:0/978080663 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb810a2700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c100930 0x7fdb7c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb818a3700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c0ff730 0x7fdb7c068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.783+0000 7fdb820a4700 1 -- 172.21.15.97:0/978080663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c0ff730 msgr2=0x7fdb7c068d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb820a4700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c0ff730 0x7fdb7c068d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb820a4700 1 -- 172.21.15.97:0/978080663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c100930 msgr2=0x7fdb7c19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb820a4700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c100930 0x7fdb7c19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb820a4700 1 -- 172.21.15.97:0/978080663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74009d00 con 0x7fdb7c1012d0 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb820a4700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009850 tx=0x7fdb740094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400e040 con 0x7fdb7c1012d0 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb74003e10 con 0x7fdb7c1012d0 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7401b950 con 0x7fdb7c1012d0 2023-10-08T09:49:01.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c1a0a10 con 0x7fdb7c1012d0 2023-10-08T09:49:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.784+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c1a0dd0 con 0x7fdb7c1012d0 2023-10-08T09:49:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.785+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c106b70 con 0x7fdb7c1012d0 2023-10-08T09:49:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.786+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fdb74005090 con 0x7fdb7c1012d0 2023-10-08T09:49:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.787+0000 7fdb72ffd700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 0x7fdb680a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.787+0000 7fdb818a3700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 0x7fdb680a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.787+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb740cce60 con 0x7fdb7c1012d0 2023-10-08T09:49:01.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.787+0000 7fdb818a3700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 0x7fdb680a3930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c100790 tx=0x7fdb7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.794+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb74098220 con 0x7fdb7c1012d0 2023-10-08T09:49:01.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/2197048595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e41023e0 msgr2=0x7fd4e4108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/2197048595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e41023e0 0x7fd4e4108c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00b0d0 tx=0x7fd4dc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/2197048595 shutdown_connections 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/2197048595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e41023e0 0x7fd4e4108c90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/2197048595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e4100840 0x7fd4e4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/2197048595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4101a40 0x7fd4e4101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/2197048595 >> 172.21.15.97:0/2197048595 conn(0x7fd4e40fbe00 msgr2=0x7fd4e40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/2197048595 shutdown_connections 2023-10-08T09:49:01.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.875+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/2197048595 wait complete. 2023-10-08T09:49:01.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 Processor -- start 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 -- start start 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4100840 0x7fd4e41967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 0x7fd4e419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e419e280 con 0x7fd4e4101a40 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e419e3c0 con 0x7fd4e41023e0 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4eb2fa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e419e550 con 0x7fd4e4100840 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4e8895700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4e8895700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54176/0 (socket says 172.21.15.97:54176) 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4e8895700 1 -- 172.21.15.97:0/3908712542 learned_addr learned my addr 172.21.15.97:0/3908712542 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:01.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4e9096700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4100840 0x7fd4e41967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.876+0000 7fd4e9897700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 0x7fd4e419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 -- 172.21.15.97:0/3908712542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4100840 msgr2=0x7fd4e41967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4100840 0x7fd4e41967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 -- 172.21.15.97:0/3908712542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 msgr2=0x7fd4e419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 0x7fd4e419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 -- 172.21.15.97:0/3908712542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc009d00 con 0x7fd4e4101a40 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e9897700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 0x7fd4e419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4e8895700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9d0 tx=0x7fd4d400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4009790 con 0x7fd4e4101a40 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4d400cbd0 con 0x7fd4e4101a40 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e419e830 con 0x7fd4e4101a40 2023-10-08T09:49:01.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4018680 con 0x7fd4e4101a40 2023-10-08T09:49:01.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.877+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e419ed30 con 0x7fd4e4101a40 2023-10-08T09:49:01.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.879+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e4107c20 con 0x7fd4e4101a40 2023-10-08T09:49:01.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.879+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fd4d4010430 con 0x7fd4e4101a40 2023-10-08T09:49:01.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.880+0000 7fd4da7fc700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 0x7fd4d00a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:01.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.880+0000 7fd4e9096700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 0x7fd4d00a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:01.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.880+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4d4014070 con 0x7fd4e4101a40 2023-10-08T09:49:01.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.880+0000 7fd4e9096700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 0x7fd4d00a3a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4e41018a0 tx=0x7fd4e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:01.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:01.889+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd4d4096460 con 0x7fd4e4101a40 2023-10-08T09:49:02.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.012+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7fdb7c10ae20 con 0x7fdb7c1012d0 2023-10-08T09:49:02.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.017+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdb7409b840 con 0x7fdb7c1012d0 2023-10-08T09:49:02.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.018+0000 7fdb72ffd700 1 -- 172.21.15.97:0/978080663 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb74009b80 con 0x7fdb7c1012d0 2023-10-08T09:49:02.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.019+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d40ca9c0 con 0x7fd4e4101a40 2023-10-08T09:49:02.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 msgr2=0x7fdb680a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 0x7fdb680a3930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c100790 tx=0x7fdb7800b410 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 msgr2=0x7fdb7c19fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009850 tx=0x7fdb740094c0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 shutdown_connections 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb680a1470 0x7fdb680a3930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb7c1012d0 0x7fdb7c19fdb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb7c0ff730 0x7fdb7c068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 --2- 172.21.15.97:0/978080663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb7c100930 0x7fdb7c19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 >> 172.21.15.97:0/978080663 conn(0x7fdb7c0facf0 msgr2=0x7fdb7c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:02.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 shutdown_connections 2023-10-08T09:49:02.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.020+0000 7fdb83b07700 1 -- 172.21.15.97:0/978080663 wait complete. 2023-10-08T09:49:02.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.114+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fd4e410c040 con 0x7fd4e4101a40 2023-10-08T09:49:02.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.117+0000 7fd4da7fc700 1 -- 172.21.15.97:0/3908712542 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v28)=0 v28) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd4d4099a80 con 0x7fd4e4101a40 2023-10-08T09:49:02.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 msgr2=0x7fd4d00a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 0x7fd4d00a3a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4e41018a0 tx=0x7fd4e0009260 comp rx=0 tx=0).stop 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 msgr2=0x7fd4e4196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9d0 tx=0x7fd4d400ece0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 shutdown_connections 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd4d00a1540 0x7fd4d00a3a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e41023e0 0x7fd4e419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e4100840 0x7fd4e41967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 --2- 172.21.15.97:0/3908712542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4e4101a40 0x7fd4e4196d20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:02.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 >> 172.21.15.97:0/3908712542 conn(0x7fd4e40fbe00 msgr2=0x7fd4e4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:02.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 shutdown_connections 2023-10-08T09:49:02.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:02.120+0000 7fd4eb2fa700 1 -- 172.21.15.97:0/3908712542 wait complete. 2023-10-08T09:49:02.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/154312241' entity='client.admin' 2023-10-08T09:49:02.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:02 smithi097 ceph-mon[99491]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:02.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/978080663' entity='client.admin' 2023-10-08T09:49:02.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3908712542' entity='client.admin' 2023-10-08T09:49:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/154312241' entity='client.admin' 2023-10-08T09:49:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:02 smithi186 ceph-mon[104393]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/978080663' entity='client.admin' 2023-10-08T09:49:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3908712542' entity='client.admin' 2023-10-08T09:49:03.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/154312241' entity='client.admin' 2023-10-08T09:49:03.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:02 smithi178 ceph-mon[103156]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:03.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/978080663' entity='client.admin' 2023-10-08T09:49:03.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3908712542' entity='client.admin' 2023-10-08T09:49:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 -- 172.21.15.97:0/1962197585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 msgr2=0x7f267c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 --2- 172.21.15.97:0/1962197585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c107b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f267400b0d0 tx=0x7f267400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 -- 172.21.15.97:0/1962197585 shutdown_connections 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 --2- 172.21.15.97:0/1962197585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c107b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 --2- 172.21.15.97:0/1962197585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c0ff730 0x7f267c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 --2- 172.21.15.97:0/1962197585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c100930 0x7f267c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 -- 172.21.15.97:0/1962197585 >> 172.21.15.97:0/1962197585 conn(0x7f267c0facf0 msgr2=0x7f267c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 -- 172.21.15.97:0/1962197585 shutdown_connections 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.799+0000 7f2682faa700 1 -- 172.21.15.97:0/1962197585 wait complete. 2023-10-08T09:49:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 Processor -- start 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 -- start start 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c0ff730 0x7f267c068cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 0x7f267c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19e2f0 con 0x7f267c1012d0 2023-10-08T09:49:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19e460 con 0x7f267c0ff730 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2682faa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19e620 con 0x7f267c100930 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54190/0 (socket says 172.21.15.97:54190) 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 -- 172.21.15.97:0/4060052267 learned_addr learned my addr 172.21.15.97:0/4060052267 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 -- 172.21.15.97:0/4060052267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 msgr2=0x7f267c19b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f267bfff700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 0x7f267c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2680d46700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c0ff730 0x7f267c068cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 0x7f267c19b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 -- 172.21.15.97:0/4060052267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c0ff730 msgr2=0x7f267c068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c0ff730 0x7f267c068cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 -- 172.21.15.97:0/4060052267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2674009d00 con 0x7f267c1012d0 2023-10-08T09:49:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f267bfff700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 0x7f267c19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2681547700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2674007bd0 tx=0x7f26740094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.800+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400e040 con 0x7f267c1012d0 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.801+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2674012dc0 con 0x7f267c1012d0 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.801+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c19e8a0 con 0x7f267c1012d0 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.801+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267401b450 con 0x7f267c1012d0 2023-10-08T09:49:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.801+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c19ed10 con 0x7f267c1012d0 2023-10-08T09:49:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.802+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c106af0 con 0x7f267c1012d0 2023-10-08T09:49:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.803+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f267401d610 con 0x7f267c1012d0 2023-10-08T09:49:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.803+0000 7f2679ffb700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 0x7f266c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.804+0000 7f2680d46700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 0x7f266c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.804+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f26740cd620 con 0x7f267c1012d0 2023-10-08T09:49:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.804+0000 7f2680d46700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 0x7f266c0a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2670005950 tx=0x7f26700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:03.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:03.807+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f26740989e0 con 0x7f267c1012d0 2023-10-08T09:49:03.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.947 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.948 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi097') 2023-10-08T09:49:03.949 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi097') 2023-10-08T09:49:04.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.021+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f267c04e4c0 con 0x7f267c1012d0 2023-10-08T09:49:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.026+0000 7f2679ffb700 1 -- 172.21.15.97:0/4060052267 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) v1 ==== 121+0+0 (secure 0 0 0) 0x7f267409c000 con 0x7f267c1012d0 2023-10-08T09:49:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 msgr2=0x7f266c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 0x7f266c0a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2670005950 tx=0x7f26700058e0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 msgr2=0x7f267c19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2674007bd0 tx=0x7f26740094f0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 shutdown_connections 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f266c0a13f0 0x7f266c0a38b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f267c1012d0 0x7f267c19db50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.028+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f267c0ff730 0x7f267c068cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.029+0000 7f2682faa700 1 --2- 172.21.15.97:0/4060052267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f267c100930 0x7f267c19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.029+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 >> 172.21.15.97:0/4060052267 conn(0x7f267c0facf0 msgr2=0x7f267c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.029+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 shutdown_connections 2023-10-08T09:49:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.029+0000 7f2682faa700 1 -- 172.21.15.97:0/4060052267 wait complete. 2023-10-08T09:49:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 -- 172.21.15.97:0/778224914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc102530 msgr2=0x7f61fc108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 --2- 172.21.15.97:0/778224914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc102530 0x7f61fc108de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00b0a0 tx=0x7f61ec00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 -- 172.21.15.97:0/778224914 shutdown_connections 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 --2- 172.21.15.97:0/778224914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc102530 0x7f61fc108de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 --2- 172.21.15.97:0/778224914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc0feb10 0x7f61fc0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 --2- 172.21.15.97:0/778224914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc101b90 0x7f61fc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 -- 172.21.15.97:0/778224914 >> 172.21.15.97:0/778224914 conn(0x7f61fc0fa450 msgr2=0x7f61fc0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 -- 172.21.15.97:0/778224914 shutdown_connections 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.190+0000 7f620432e700 1 -- 172.21.15.97:0/778224914 wait complete. 2023-10-08T09:49:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 Processor -- start 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 -- start start 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc101b90 0x7f61fc196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc102530 0x7f61fc19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc19e350 con 0x7f61fc0feb10 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc19e490 con 0x7f61fc102530 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc19e620 con 0x7f61fc101b90 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54200/0 (socket says 172.21.15.97:54200) 2023-10-08T09:49:04.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 -- 172.21.15.97:0/273947409 learned_addr learned my addr 172.21.15.97:0/273947409 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62018c9700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc101b90 0x7f61fc196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 -- 172.21.15.97:0/273947409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc101b90 msgr2=0x7f61fc196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc101b90 0x7f61fc196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 -- 172.21.15.97:0/273947409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc102530 msgr2=0x7f61fc19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc102530 0x7f61fc19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 -- 172.21.15.97:0/273947409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec009d00 con 0x7f61fc0feb10 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f62020ca700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61f400d9b0 tx=0x7f61f400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400be80 con 0x7f61fc0feb10 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61f40044e0 con 0x7f61fc0feb10 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.191+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc19e900 con 0x7f61fc0feb10 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.192+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4003b40 con 0x7f61fc0feb10 2023-10-08T09:49:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.192+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc19ee00 con 0x7f61fc0feb10 2023-10-08T09:49:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.193+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc107df0 con 0x7f61fc0feb10 2023-10-08T09:49:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.194+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f61f4009940 con 0x7f61fc0feb10 2023-10-08T09:49:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.194+0000 7f61fb7fe700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 0x7f61e40a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.194+0000 7f62018c9700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 0x7f61e40a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.195+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61f40cad80 con 0x7f61fc0feb10 2023-10-08T09:49:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.195+0000 7f62018c9700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 0x7f61e40a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005fd0 tx=0x7f61f000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:04.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.199+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f61f40960b0 con 0x7f61fc0feb10 2023-10-08T09:49:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.389+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f61fc04e4c0 con 0x7f61fc0feb10 2023-10-08T09:49:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.395+0000 7f61fb7fe700 1 -- 172.21.15.97:0/273947409 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v30)=0 v30) v1 ==== 120+0+0 (secure 0 0 0) 0x7f61f40996d0 con 0x7f61fc0feb10 2023-10-08T09:49:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 msgr2=0x7f61e40a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 0x7f61e40a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005fd0 tx=0x7f61f000a560 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 msgr2=0x7f61fc196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61f400d9b0 tx=0x7f61f400dd70 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 shutdown_connections 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f61e40a13f0 0x7f61e40a38b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc102530 0x7f61fc19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61fc0feb10 0x7f61fc196810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 --2- 172.21.15.97:0/273947409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61fc101b90 0x7f61fc196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 >> 172.21.15.97:0/273947409 conn(0x7f61fc0fa450 msgr2=0x7f61fc104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 shutdown_connections 2023-10-08T09:49:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:04.397+0000 7f620432e700 1 -- 172.21.15.97:0/273947409 wait complete. 2023-10-08T09:49:04.893 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi097') 2023-10-08T09:49:04.893 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '7ec07914-65bf-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2023-10-08T09:49:04.894 INFO:tasks.cephadm:Deploying OSDs... 2023-10-08T09:49:04.894 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:49:04.894 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:49:04.912 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:49:04.913 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-10-08T09:49:04.970 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:49:04.970 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 394885 Links: 1 2023-10-08T09:49:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-10-08 09:47:55.934893151 +0000 2023-10-08T09:49:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-10-08 09:42:35.645513507 +0000 2023-10-08T09:49:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-10-08 09:42:35.645513507 +0000 2023-10-08T09:49:04.971 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-10-08T09:49:04.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:49:05.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-10-08T09:49:05.039 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-10-08T09:49:05.039 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000261318 s, 2.0 MB/s 2023-10-08T09:49:05.040 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:49:05.098 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 394959 Links: 1 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:05.156 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-10-08 09:47:55.987892262 +0000 2023-10-08T09:49:05.157 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-10-08 09:42:36.165504705 +0000 2023-10-08T09:49:05.157 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-10-08 09:42:36.165504705 +0000 2023-10-08T09:49:05.157 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-10-08T09:49:05.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:49:05.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:05 smithi097 ceph-mon[99491]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:05.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:05 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4060052267' entity='client.admin' 2023-10-08T09:49:05.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:05 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/273947409' entity='client.admin' 2023-10-08T09:49:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-10-08T09:49:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-10-08T09:49:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000226275 s, 2.3 MB/s 2023-10-08T09:49:05.223 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:49:05.281 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 395056 Links: 1 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-10-08 09:47:56.044891306 +0000 2023-10-08T09:49:05.339 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-10-08 09:42:36.665496239 +0000 2023-10-08T09:49:05.339 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-10-08 09:42:36.665496239 +0000 2023-10-08T09:49:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-10-08T09:49:05.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:49:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-10-08T09:49:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-10-08T09:49:05.403 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000210164 s, 2.4 MB/s 2023-10-08T09:49:05.404 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:49:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:05 smithi178 ceph-mon[103156]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:05 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4060052267' entity='client.admin' 2023-10-08T09:49:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:05 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/273947409' entity='client.admin' 2023-10-08T09:49:05.462 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-10-08T09:49:05.520 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:49:05.520 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:05.520 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 384586 Links: 1 2023-10-08T09:49:05.520 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:05.521 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:05.521 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-10-08 09:47:56.113890148 +0000 2023-10-08T09:49:05.521 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-10-08 09:42:37.173487637 +0000 2023-10-08T09:49:05.521 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-10-08 09:42:37.173487637 +0000 2023-10-08T09:49:05.521 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-10-08T09:49:05.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:49:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:05 smithi186 ceph-mon[104393]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:05 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4060052267' entity='client.admin' 2023-10-08T09:49:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:05 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/273947409' entity='client.admin' 2023-10-08T09:49:05.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-10-08T09:49:05.585 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-10-08T09:49:05.585 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000221758 s, 2.3 MB/s 2023-10-08T09:49:05.587 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:49:05.644 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-08T09:49:05.644 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:49:05.665 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:49:05.666 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 390302 Links: 1 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-08 09:48:46.311374851 +0000 2023-10-08T09:49:05.728 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-08 09:42:45.889253507 +0000 2023-10-08T09:49:05.729 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-08 09:42:45.889253507 +0000 2023-10-08T09:49:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-10-08T09:49:05.729 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:49:05.802 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-08T09:49:05.803 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-08T09:49:05.803 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000139717 s, 3.7 MB/s 2023-10-08T09:49:05.804 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:49:05.865 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2023-10-08T09:49:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 390418 Links: 1 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-08 09:48:46.311374851 +0000 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-08 09:42:46.427243787 +0000 2023-10-08T09:49:05.925 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-08 09:42:46.427243787 +0000 2023-10-08T09:49:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-10-08T09:49:05.926 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:49:05.993 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-08T09:49:05.993 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-08T09:49:05.993 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000163767 s, 3.1 MB/s 2023-10-08T09:49:05.994 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:49:06.053 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2023-10-08T09:49:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 394679 Links: 1 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-08 09:48:46.311374851 +0000 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-08 09:42:46.940234518 +0000 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-08 09:42:46.940234518 +0000 2023-10-08T09:49:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-10-08T09:49:06.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:49:06.179 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-08T09:49:06.179 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-08T09:49:06.179 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.5991e-05 s, 5.3 MB/s 2023-10-08T09:49:06.180 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:49:06.239 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 393750 Links: 1 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-08 09:48:46.312374833 +0000 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-08 09:42:47.438225520 +0000 2023-10-08T09:49:06.297 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-08 09:42:47.438225520 +0000 2023-10-08T09:49:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-10-08T09:49:06.298 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:49:06.363 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-08T09:49:06.363 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-08T09:49:06.363 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000108214 s, 4.7 MB/s 2023-10-08T09:49:06.364 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:49:06.422 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-08T09:49:06.422 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:49:06.444 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:49:06.445 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-10-08T09:49:06.506 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:49:06.506 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 388592 Links: 1 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-08 09:48:44.081415775 +0000 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-08 09:42:48.691043322 +0000 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-08 09:42:48.691043322 +0000 2023-10-08T09:49:06.507 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-08T09:49:06.507 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:49:06.580 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-08T09:49:06.581 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-08T09:49:06.581 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000264275 s, 1.9 MB/s 2023-10-08T09:49:06.582 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:49:06.642 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 393041 Links: 1 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-08 09:48:44.081415775 +0000 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-08 09:42:49.197033987 +0000 2023-10-08T09:49:06.701 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-08 09:42:49.197033987 +0000 2023-10-08T09:49:06.702 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-08T09:49:06.702 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:49:06.769 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-08T09:49:06.769 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-08T09:49:06.769 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000232268 s, 2.2 MB/s 2023-10-08T09:49:06.770 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:49:06.829 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 386603 Links: 1 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-08 09:48:44.081415775 +0000 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-08 09:42:49.706024597 +0000 2023-10-08T09:49:06.887 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-08 09:42:49.706024597 +0000 2023-10-08T09:49:06.888 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-08T09:49:06.888 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:49:06.956 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-08T09:49:06.956 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-08T09:49:06.956 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000240946 s, 2.1 MB/s 2023-10-08T09:49:06.957 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:49:07.015 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 386648 Links: 1 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-08 09:48:44.081415775 +0000 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-08 09:42:50.218015152 +0000 2023-10-08T09:49:07.073 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-08 09:42:50.218015152 +0000 2023-10-08T09:49:07.074 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-08T09:49:07.074 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:49:07.139 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-08T09:49:07.139 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-08T09:49:07.139 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000177962 s, 2.9 MB/s 2023-10-08T09:49:07.140 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:49:07.198 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2023-10-08T09:49:07.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:49:07.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:07 smithi186 ceph-mon[104393]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:07.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:07 smithi097 ceph-mon[99491]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:07 smithi178 ceph-mon[103156]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:07.662 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:49:09.154 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:49:09.155 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:49:09.155 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-10-08T09:49:09.155 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-10-08T09:49:09.155 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110212 s, 951 MB/s 2023-10-08T09:49:09.155 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-10-08T09:49:09.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:09 smithi097 ceph-mon[99491]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:09 smithi178 ceph-mon[103156]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:09.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:09 smithi186 ceph-mon[104393]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:09.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2023-10-08T09:49:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:49:11.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:11 smithi097 ceph-mon[99491]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 -- 172.21.15.97:0/3192945774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec102470 msgr2=0x7fa6ec108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/3192945774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec102470 0x7fa6ec108d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b0a0 tx=0x7fa6e400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 -- 172.21.15.97:0/3192945774 shutdown_connections 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/3192945774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec102470 0x7fa6ec108d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/3192945774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/3192945774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 0x7fa6ec101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 -- 172.21.15.97:0/3192945774 >> 172.21.15.97:0/3192945774 conn(0x7fa6ec0fa450 msgr2=0x7fa6ec0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 -- 172.21.15.97:0/3192945774 shutdown_connections 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.356+0000 7fa6f34bd700 1 -- 172.21.15.97:0/3192945774 wait complete. 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 Processor -- start 2023-10-08T09:49:11.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 -- start start 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 0x7fa6ec0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 0x7fa6ec0fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec100760 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1008d0 con 0x7fa6ec101ad0 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f34bd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1a08f0 con 0x7fa6ec102470 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54212/0 (socket says 172.21.15.97:54212) 2023-10-08T09:49:11.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 -- 172.21.15.97:0/579170273 learned_addr learned my addr 172.21.15.97:0/579170273 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 -- 172.21.15.97:0/579170273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 msgr2=0x7fa6ec0fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1a5a700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 0x7fa6ec0fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 0x7fa6ec0fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f0a58700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 0x7fa6ec0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 -- 172.21.15.97:0/579170273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 msgr2=0x7fa6ec0ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 0x7fa6ec0ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 -- 172.21.15.97:0/579170273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009d00 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1a5a700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 0x7fa6ec0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.357+0000 7fa6f1259700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c970 tx=0x7fa6dc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.358+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc004220 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.358+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6dc007d60 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.358+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec1a0b80 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.358+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc003d00 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.358+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec1a10d0 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.359+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec107cb0 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.360+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fa6dc004380 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.360+0000 7fa6e27fc700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 0x7fa6d80a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.360+0000 7fa6f0a58700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 0x7fa6d80a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.360+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa6dc0caef0 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.361+0000 7fa6f0a58700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 0x7fa6d80a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:11.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.364+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa6dc096220 con 0x7fa6ec0feaf0 2023-10-08T09:49:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:11 smithi178 ceph-mon[103156]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:11 smithi186 ceph-mon[104393]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:11.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:11.563+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6ec02cf80 con 0x7fa6d80a1470 2023-10-08T09:49:12.382 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:12.383 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:12.383 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:13 smithi097 ceph-mon[99491]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:13 smithi097 ceph-mon[99491]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:13 smithi178 ceph-mon[103156]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:13 smithi178 ceph-mon[103156]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:13 smithi186 ceph-mon[104393]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:13 smithi186 ceph-mon[104393]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:14.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:14 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]: dispatch 2023-10-08T09:49:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:14 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]': finished 2023-10-08T09:49:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:14 smithi097 ceph-mon[99491]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:49:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:14 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]: dispatch 2023-10-08T09:49:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:14 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]': finished 2023-10-08T09:49:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:14 smithi178 ceph-mon[103156]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:49:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:14 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]: dispatch 2023-10-08T09:49:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:14 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3627876021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31b0d724-2bfc-4c58-8ac5-38962f5d300a"}]': finished 2023-10-08T09:49:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:14 smithi186 ceph-mon[104393]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:49:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:15 smithi097 ceph-mon[99491]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4257668250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:15 smithi178 ceph-mon[103156]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4257668250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:15 smithi186 ceph-mon[104393]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4257668250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:17 smithi097 ceph-mon[99491]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:17 smithi178 ceph-mon[103156]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:17 smithi186 ceph-mon[104393]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:19 smithi097 ceph-mon[99491]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:19 smithi178 ceph-mon[103156]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:19 smithi186 ceph-mon[104393]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:21.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:21 smithi097 ceph-mon[99491]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:21.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:21 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:49:21.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:21 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:21 smithi178 ceph-mon[103156]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:21 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:49:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:21 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:21 smithi186 ceph-mon[104393]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:21 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:49:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:21 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:22.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:22 smithi097 ceph-mon[99491]: Deploying daemon osd.0 on smithi097 2023-10-08T09:49:22.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:22 smithi178 ceph-mon[103156]: Deploying daemon osd.0 on smithi097 2023-10-08T09:49:22.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:22 smithi186 ceph-mon[104393]: Deploying daemon osd.0 on smithi097 2023-10-08T09:49:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:23 smithi097 ceph-mon[99491]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:23 smithi178 ceph-mon[103156]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:23 smithi186 ceph-mon[104393]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:25.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:25 smithi097 ceph-mon[99491]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:25.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:25 smithi178 ceph-mon[103156]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:25 smithi186 ceph-mon[104393]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:27.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.214+0000 7fa6e27fc700 1 -- 172.21.15.97:0/579170273 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6ec02cf80 con 0x7fa6d80a1470 2023-10-08T09:49:27.215 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2023-10-08T09:49:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.216+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 msgr2=0x7fa6d80a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:27.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.216+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 0x7fa6d80a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.216+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 msgr2=0x7fa6ec1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.216+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c970 tx=0x7fa6dc00cd30 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 shutdown_connections 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6d80a1470 0x7fa6d80a3930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6ec102470 0x7fa6ec0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0feaf0 0x7fa6ec1013e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 --2- 172.21.15.97:0/579170273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101ad0 0x7fa6ec0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 >> 172.21.15.97:0/579170273 conn(0x7fa6ec0fa450 msgr2=0x7fa6ec104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 shutdown_connections 2023-10-08T09:49:27.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:49:27.217+0000 7fa6f34bd700 1 -- 172.21.15.97:0/579170273 wait complete. 2023-10-08T09:49:27.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:27.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.375 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:27.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:27.376 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:27.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:27.654 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-08T09:49:27.657 INFO:tasks.cephadm:Deploying osd.1 on smithi178 with /dev/vg_nvme/lv_4... 2023-10-08T09:49:27.657 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:49:27.688 INFO:journalctl@ceph.osd.0.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:49:28.221 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:49:28.441 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:28 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.442 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:28 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.442 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:28 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:28.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:28 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:28 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:28 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:28 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:28 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:28 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:28.702 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:49:28 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T09:49:28.626+0000 7fed45470640 -1 osd.0 0 log_to_monitors true 2023-10-08T09:49:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:29 smithi097 ceph-mon[99491]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:29 smithi097 ceph-mon[99491]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:49:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:29 smithi178 ceph-mon[103156]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:29 smithi178 ceph-mon[103156]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:49:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:29 smithi186 ceph-mon[104393]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:29 smithi186 ceph-mon[104393]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:49:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:49:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:49:29.761 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-08T09:49:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-08T09:49:29.761 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00831584 s, 1.3 GB/s 2023-10-08T09:49:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-08T09:49:30.179 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2023-10-08T09:49:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:30 smithi178 ceph-mon[103156]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:49:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:30 smithi178 ceph-mon[103156]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:49:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:30 smithi178 ceph-mon[103156]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:49:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:30 smithi178 ceph-mon[103156]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:30 smithi186 ceph-mon[104393]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:49:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:30 smithi186 ceph-mon[104393]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:49:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:30 smithi186 ceph-mon[104393]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:49:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:30 smithi186 ceph-mon[104393]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:30.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T09:49:30.201+0000 7fed3aae7700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T09:49:30.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T09:49:30.204+0000 7fed340d7700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:49:30.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-mon[99491]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:49:30.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-mon[99491]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:49:30.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-mon[99491]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:49:30.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:30.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:30 smithi097 ceph-mon[99491]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:30.733 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi097 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:31.477 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi097 to 17168M 2023-10-08T09:49:31.478 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.478 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:31.478 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:31.478 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.478 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi097 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi097 to 17168M 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi097 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi097 to 17168M 2023-10-08T09:49:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:31.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:31.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:31.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 -- 172.21.15.178:0/1938254355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c70100970 msgr2=0x7f7c70100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 --2- 172.21.15.178:0/1938254355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c70100970 0x7f7c70100df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009990 tx=0x7f7c60009ca0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 -- 172.21.15.178:0/1938254355 shutdown_connections 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 --2- 172.21.15.178:0/1938254355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70101330 0x7f7c70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 --2- 172.21.15.178:0/1938254355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c700ffc50 0x7f7c70100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 --2- 172.21.15.178:0/1938254355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c70100970 0x7f7c70100df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 -- 172.21.15.178:0/1938254355 >> 172.21.15.178:0/1938254355 conn(0x7f7c700fa450 msgr2=0x7f7c700fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 -- 172.21.15.178:0/1938254355 shutdown_connections 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.975+0000 7f7c75a80700 1 -- 172.21.15.178:0/1938254355 wait complete. 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 Processor -- start 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 -- start start 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 0x7f7c70195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c70101330 0x7f7c7019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7019d250 con 0x7f7c70100970 2023-10-08T09:49:31.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7019d390 con 0x7f7c700ffc50 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c75a80700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7019d520 con 0x7f7c70101330 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45724/0 (socket says 172.21.15.178:45724) 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 -- 172.21.15.178:0/250104663 learned_addr learned my addr 172.21.15.178:0/250104663 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6effd700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 0x7f7c70195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6ffff700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c70101330 0x7f7c7019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 -- 172.21.15.178:0/250104663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c70101330 msgr2=0x7f7c7019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c70101330 0x7f7c7019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 -- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 msgr2=0x7f7c70195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 0x7f7c70195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6f7fe700 1 -- 172.21.15.178:0/250104663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600095f0 con 0x7f7c700ffc50 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.976+0000 7f7c6effd700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 0x7f7c70195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c6f7fe700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400d980 tx=0x7f7c6400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400de60 con 0x7f7c700ffc50 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7019d800 con 0x7f7c700ffc50 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c640044e0 con 0x7f7c700ffc50 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003bc0 con 0x7f7c700ffc50 2023-10-08T09:49:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.977+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7019dd50 con 0x7f7c700ffc50 2023-10-08T09:49:31.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.978+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c70106c20 con 0x7f7c700ffc50 2023-10-08T09:49:31.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.979+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f7c64009bb0 con 0x7f7c700ffc50 2023-10-08T09:49:31.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.979+0000 7f7c6cff9700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 0x7f7c580a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:31.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.979+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f7c640cb2e0 con 0x7f7c700ffc50 2023-10-08T09:49:31.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.980+0000 7f7c6effd700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 0x7f7c580a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:31.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.980+0000 7f7c6effd700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 0x7f7c580a38b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b5c0 tx=0x7f7c60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:31.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:31.983+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7c640963b0 con 0x7f7c700ffc50 2023-10-08T09:49:32.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:32.178+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c7002cf80 con 0x7f7c580a13f0 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] boot 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:32.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] boot 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: osd.0 [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] boot 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:49:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:33.301 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:33 smithi178 ceph-mon[103156]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:49:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:33 smithi186 ceph-mon[104393]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:49:33.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:33 smithi097 ceph-mon[99491]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:49:34.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:34 smithi186 ceph-mon[104393]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:34.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:34 smithi097 ceph-mon[99491]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:34.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:34 smithi178 ceph-mon[103156]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/3173200738' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]': finished 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:35 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/720778198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/3173200738' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]': finished 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:35 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/720778198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/3173200738' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]: dispatch 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "946de8df-41b4-46e5-87da-4822fa096d12"}]': finished 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:35 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/720778198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:36.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:36 smithi186 ceph-mon[104393]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:36.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:36 smithi097 ceph-mon[99491]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:36 smithi178 ceph-mon[103156]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:39.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:38 smithi097 ceph-mon[99491]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:38 smithi178 ceph-mon[103156]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:38 smithi186 ceph-mon[104393]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:41 smithi097 ceph-mon[99491]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:41 smithi178 ceph-mon[103156]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:41 smithi186 ceph-mon[104393]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:49:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:42 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:42.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:42 smithi097 ceph-mon[99491]: Deploying daemon osd.1 on smithi178 2023-10-08T09:49:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:49:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:42 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:42.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:42 smithi178 ceph-mon[103156]: Deploying daemon osd.1 on smithi178 2023-10-08T09:49:42.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:49:42.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:42 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:42.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:42 smithi186 ceph-mon[104393]: Deploying daemon osd.1 on smithi178 2023-10-08T09:49:43.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:43 smithi178 ceph-mon[103156]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:43.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:43 smithi097 ceph-mon[99491]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:43.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:43 smithi186 ceph-mon[104393]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:45.331 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:45 smithi178 ceph-mon[103156]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:45.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:45 smithi097 ceph-mon[99491]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:45 smithi186 ceph-mon[104393]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:46.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:46.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:46 smithi097 ceph-mon[99491]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:46 smithi178 ceph-mon[103156]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:46.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:46.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:46.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:46 smithi186 ceph-mon[104393]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:48.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.537+0000 7f7c6cff9700 1 -- 172.21.15.178:0/250104663 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7c7002cf80 con 0x7f7c580a13f0 2023-10-08T09:49:48.538 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 1 on host 'smithi178' 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 msgr2=0x7f7c580a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 0x7f7c580a38b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b5c0 tx=0x7f7c60005fb0 comp rx=0 tx=0).stop 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 msgr2=0x7f7c70195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400d980 tx=0x7f7c6400dd40 comp rx=0 tx=0).stop 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 shutdown_connections 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7c580a13f0 0x7f7c580a38b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:48.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c70101330 0x7f7c7019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:48.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c700ffc50 0x7f7c70195710 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:48.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 --2- 172.21.15.178:0/250104663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c70100970 0x7f7c70195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:48.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.539+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 >> 172.21.15.178:0/250104663 conn(0x7f7c700fa450 msgr2=0x7f7c70103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:48.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.540+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 shutdown_connections 2023-10-08T09:49:48.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:49:48.540+0000 7f7c75a80700 1 -- 172.21.15.178:0/250104663 wait complete. 2023-10-08T09:49:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:48.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:49.024 DEBUG:teuthology.orchestra.run.smithi178:osd.1> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:49.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:49.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:49.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:49.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:49:49.027 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2023-10-08T09:49:49.028 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:49:49.060 INFO:journalctl@ceph.osd.1.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:49:49.593 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:49:49.705 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:49:49 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T09:49:49.383+0000 7f03e0e8e640 -1 osd.1 0 log_to_monitors true 2023-10-08T09:49:49.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:49 smithi178 ceph-mon[103156]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:49.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:49 smithi178 ceph-mon[103156]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:49.814 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:49 smithi186 ceph-mon[104393]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:49.814 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:49 smithi186 ceph-mon[104393]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:49.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:49 smithi097 ceph-mon[99491]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:49.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:49 smithi097 ceph-mon[99491]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:49:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:49:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:49:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:50.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:50 smithi186 ceph-mon[104393]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:50.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:49:50.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:49:50.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:50.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:50 smithi097 ceph-mon[99491]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:49:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: from='osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:49:50.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:50.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:49:50.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-mon[103156]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:50.956 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T09:49:50.568+0000 7f03d6505700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T09:49:50.956 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:49:50 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T09:49:50.571+0000 7f03d0af7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:49:51.043 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:49:51.043 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:49:51.044 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-08T09:49:51.044 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-08T09:49:51.044 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110811 s, 946 MB/s 2023-10-08T09:49:51.044 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-08T09:49:51.448 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2023-10-08T09:49:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:51 smithi186 ceph-mon[104393]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:49:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:51 smithi186 ceph-mon[104393]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:49:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:51.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:51 smithi097 ceph-mon[99491]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:49:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:51 smithi097 ceph-mon[99491]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:49:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:51 smithi178 ceph-mon[103156]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:49:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:51 smithi178 ceph-mon[103156]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:49:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:52.007 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] boot 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi178 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi178 to 17168M 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:52 smithi186 ceph-mon[104393]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] boot 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi178 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi178 to 17168M 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:52 smithi097 ceph-mon[99491]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: osd.1 [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] boot 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:49:52.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi178 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi178 to 17168M 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:49:52.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:52 smithi178 ceph-mon[103156]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 -- 172.21.15.186:0/2094259675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac100050 msgr2=0x7f2eac1004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2094259675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac100050 0x7f2eac1004d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800aaa0 tx=0x7f2ea800adb0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 -- 172.21.15.186:0/2094259675 shutdown_connections 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2094259675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 0x7f2eac107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2094259675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eac0fee60 0x7f2eac0ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.157 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2094259675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac100050 0x7f2eac1004d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 -- 172.21.15.186:0/2094259675 >> 172.21.15.186:0/2094259675 conn(0x7f2eac0fa450 msgr2=0x7f2eac0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 -- 172.21.15.186:0/2094259675 shutdown_connections 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.154+0000 7f2eb394f700 1 -- 172.21.15.186:0/2094259675 wait complete. 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 Processor -- start 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 -- start start 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eac100050 0x7f2eac196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 0x7f2eac19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac19e350 con 0x7f2eac100a10 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac19e490 con 0x7f2eac100050 2023-10-08T09:49:53.158 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb394f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac19e620 con 0x7f2eac0fee60 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:56902/0 (socket says 172.21.15.186:56902) 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 -- 172.21.15.186:0/2527120165 learned_addr learned my addr 172.21.15.186:0/2527120165 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb1eec700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 0x7f2eac19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 -- 172.21.15.186:0/2527120165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eac100050 msgr2=0x7f2eac196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eac100050 0x7f2eac196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 -- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 msgr2=0x7f2eac19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 0x7f2eac19dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 -- 172.21.15.186:0/2527120165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea800a700 con 0x7f2eac0fee60 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.155+0000 7f2eb16eb700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b530 tx=0x7f2e9c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.156+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c011830 con 0x7f2eac0fee60 2023-10-08T09:49:53.159 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.156+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e9c011e50 con 0x7f2eac0fee60 2023-10-08T09:49:53.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.156+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00f2e0 con 0x7f2eac0fee60 2023-10-08T09:49:53.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.156+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac19e900 con 0x7f2eac0fee60 2023-10-08T09:49:53.160 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.156+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac19ee80 con 0x7f2eac0fee60 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.157+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac1063b0 con 0x7f2eac0fee60 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.158+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2e9c010580 con 0x7f2eac0fee60 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.158+0000 7f2ea27fc700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 0x7f2e980a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.158+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f2e9c0caed0 con 0x7f2eac0fee60 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.158+0000 7f2eb0eea700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 0x7f2e980a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:49:53.161 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.159+0000 7f2eb0eea700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 0x7f2e980a3930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c6d0 tx=0x7f2ea8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:49:53.163 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.163+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e9c095d80 con 0x7f2eac0fee60 2023-10-08T09:49:53.360 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:49:53.360+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2eac02cf80 con 0x7f2e980a1470 2023-10-08T09:49:53.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:53 smithi097 ceph-mon[99491]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:49:53.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:53 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:53.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:53 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:53.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:53 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:53 smithi178 ceph-mon[103156]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:49:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:53 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:53 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:53 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:53 smithi186 ceph-mon[104393]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:49:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:53 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:49:54.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:53 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:49:54.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:53 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:49:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:54 smithi186 ceph-mon[104393]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:54 smithi186 ceph-mon[104393]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:54.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:54 smithi097 ceph-mon[99491]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:54.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:54 smithi097 ceph-mon[99491]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:54 smithi178 ceph-mon[103156]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:49:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:54 smithi178 ceph-mon[103156]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/3121224684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:56 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:56 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]': finished 2023-10-08T09:49:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:56 smithi097 ceph-mon[99491]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:49:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:56 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:49:56.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/3121224684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:56 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:56 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]': finished 2023-10-08T09:49:56.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:56 smithi178 ceph-mon[103156]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:49:56.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:56 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:49:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/3121224684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:56 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]: dispatch 2023-10-08T09:49:56.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:56 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dcdc682-9882-47fb-889a-4d412fbc7433"}]': finished 2023-10-08T09:49:56.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:56 smithi186 ceph-mon[104393]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:49:56.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:56 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:49:57.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:57 smithi097 ceph-mon[99491]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/1626152106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:57 smithi178 ceph-mon[103156]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/1626152106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:57 smithi186 ceph-mon[104393]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/1626152106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:49:59.394 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:49:59 smithi186 ceph-mon[104393]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:59.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:49:59 smithi097 ceph-mon[99491]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:49:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:49:59 smithi178 ceph-mon[103156]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:00.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:00 smithi186 ceph-mon[104393]: overall HEALTH_OK 2023-10-08T09:50:00.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:00 smithi097 ceph-mon[99491]: overall HEALTH_OK 2023-10-08T09:50:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:00 smithi178 ceph-mon[103156]: overall HEALTH_OK 2023-10-08T09:50:01.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:01 smithi186 ceph-mon[104393]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:01 smithi097 ceph-mon[99491]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:01 smithi178 ceph-mon[103156]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:03.196 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:03 smithi186 ceph-mon[104393]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:03.196 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:03 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:50:03.196 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:03 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:03 smithi097 ceph-mon[99491]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:03 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:50:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:03 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:03 smithi178 ceph-mon[103156]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:03 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:50:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:03 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:04 smithi097 ceph-mon[99491]: Deploying daemon osd.2 on smithi186 2023-10-08T09:50:04.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:04 smithi178 ceph-mon[103156]: Deploying daemon osd.2 on smithi186 2023-10-08T09:50:04.457 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:04 smithi186 ceph-mon[104393]: Deploying daemon osd.2 on smithi186 2023-10-08T09:50:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:05 smithi097 ceph-mon[99491]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:05 smithi178 ceph-mon[103156]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:05.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:05 smithi186 ceph-mon[104393]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:07.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:07 smithi186 ceph-mon[104393]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:07.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:07 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:07 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:07 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:07.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:07 smithi097 ceph-mon[99491]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:07.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:07 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:07 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:07 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:07 smithi178 ceph-mon[103156]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:07 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:07 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:07 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:09.074 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2023-10-08T09:50:09.075 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.072+0000 7f2ea27fc700 1 -- 172.21.15.186:0/2527120165 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2eac02cf80 con 0x7f2e980a1470 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 msgr2=0x7f2e980a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 0x7f2e980a3930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c6d0 tx=0x7f2ea8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 msgr2=0x7f2eac196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b530 tx=0x7f2e9c00b8f0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 shutdown_connections 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e980a1470 0x7f2e980a3930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.077 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eac100a10 0x7f2eac19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eac0fee60 0x7f2eac196860 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 --2- 172.21.15.186:0/2527120165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eac100050 0x7f2eac196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:09.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 >> 172.21.15.186:0/2527120165 conn(0x7f2eac0fa450 msgr2=0x7f2eac1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:09.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 shutdown_connections 2023-10-08T09:50:09.078 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:09.075+0000 7f2eb394f700 1 -- 172.21.15.186:0/2527120165 wait complete. 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:09.396 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-08T09:50:09.398 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_3... 2023-10-08T09:50:09.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:50:09.427 INFO:journalctl@ceph.osd.2.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:50:09.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:09.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:09.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:09.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.432 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:09.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:50:10.749 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:10 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T09:50:10.550+0000 7f5121efd640 -1 osd.2 0 log_to_monitors true 2023-10-08T09:50:11.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:11 smithi097 ceph-mon[99491]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:11.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:11 smithi097 ceph-mon[99491]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:11 smithi097 ceph-mon[99491]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:50:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:50:11.445 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-10-08T09:50:11.446 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-10-08T09:50:11.446 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111044 s, 944 MB/s 2023-10-08T09:50:11.446 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2023-10-08T09:50:11.446 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-10-08T09:50:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:11 smithi178 ceph-mon[103156]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:11 smithi178 ceph-mon[103156]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:11 smithi178 ceph-mon[103156]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:11 smithi186 ceph-mon[104393]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:11 smithi186 ceph-mon[104393]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:11 smithi186 ceph-mon[104393]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:50:11.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2023-10-08T09:50:12.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:50:12.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:50:12.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:12.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:12.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.276 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T09:50:12.118+0000 7f5117574700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T09:50:12.277 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:12 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T09:50:12.121+0000 7f5112367700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:50:12.430 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:50:12.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:12.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:12.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:12.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:12.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:12.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:13.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi186 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi186 2023-10-08T09:50:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-08T09:50:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:50:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:50:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi186 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:13.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- 172.21.15.97:0/1442132746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc04100930 msgr2=0x7fbc04100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/1442132746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc04100930 0x7fbc04100d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40099c0 tx=0x7fbbf4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- 172.21.15.97:0/1442132746 shutdown_connections 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/1442132746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc04107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/1442132746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc040ff730 0x7fbc040ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/1442132746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc04100930 0x7fbc04100d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- 172.21.15.97:0/1442132746 >> 172.21.15.97:0/1442132746 conn(0x7fbc040facf0 msgr2=0x7fbc040fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- 172.21.15.97:0/1442132746 shutdown_connections 2023-10-08T09:50:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- 172.21.15.97:0/1442132746 wait complete. 2023-10-08T09:50:13.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 Processor -- start 2023-10-08T09:50:13.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 -- start start 2023-10-08T09:50:13.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:13.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 0x7fbc04195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:13.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.628+0000 7fbc0c0da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc0419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0c0da700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419d1b0 con 0x7fbc040ff730 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0c0da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419d2f0 con 0x7fbc041012d0 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0c0da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419d480 con 0x7fbc04100930 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54280/0 (socket says 172.21.15.97:54280) 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 -- 172.21.15.97:0/3507452400 learned_addr learned my addr 172.21.15.97:0/3507452400 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09675700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 0x7fbc04195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0a677700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc0419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 -- 172.21.15.97:0/3507452400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 msgr2=0x7fbc04195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 0x7fbc04195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 -- 172.21.15.97:0/3507452400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 msgr2=0x7fbc0419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:13.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc0419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 -- 172.21.15.97:0/3507452400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40095f0 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09675700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 0x7fbc04195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0a677700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc0419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc09e76700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d9b0 tx=0x7fbc0000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000be80 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc000044e0 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0419d760 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003b40 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.629+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0419dcb0 con 0x7fbc040ff730 2023-10-08T09:50:13.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.630+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04106c60 con 0x7fbc040ff730 2023-10-08T09:50:13.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.631+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fbc00009940 con 0x7fbc040ff730 2023-10-08T09:50:13.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.632+0000 7fbbfaffd700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 0x7fbbf00a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:13.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.632+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fbc000cb710 con 0x7fbc040ff730 2023-10-08T09:50:13.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.632+0000 7fbc09675700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 0x7fbbf00a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:13.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.632+0000 7fbc09675700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 0x7fbbf00a3930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4006010 tx=0x7fbbf4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:13.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.635+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc00096420 con 0x7fbc040ff730 2023-10-08T09:50:13.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:13.833+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc0402cf80 con 0x7fbbf00a1470 2023-10-08T09:50:14.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] boot 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] boot 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: osd.2 [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] boot 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:15 smithi097 ceph-mon[99491]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:15 smithi097 ceph-mon[99491]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:15 smithi097 ceph-mon[99491]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:50:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:15 smithi178 ceph-mon[103156]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:15 smithi178 ceph-mon[103156]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:15 smithi178 ceph-mon[103156]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:50:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:15 smithi186 ceph-mon[104393]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:50:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:15 smithi186 ceph-mon[104393]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:15 smithi186 ceph-mon[104393]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:50:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:50:15.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128984]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:15.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:15.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128984]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:15.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128984]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:16.144 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128999]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:16.145 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:16.145 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128999]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:16.145 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 09:50:15 smithi097 sudo[128999]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:16.294 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:16.294 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:16 smithi178 ceph-mon[103156]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:50:16.294 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:16.294 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113704]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:16.295 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113704]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:16.295 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113704]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:16.295 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113704]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:16.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:16.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:16 smithi097 ceph-mon[99491]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:50:16.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:50:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:16 smithi186 ceph-mon[104393]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:50:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:16.705 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113707]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:16.705 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:16.705 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113707]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:16.705 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 09:50:16 smithi178 sudo[113707]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:16.816 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115616]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:16.817 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:16.817 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115616]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:16.817 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115616]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:17.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115619]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:17.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:17.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115619]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:17.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 09:50:16 smithi186 sudo[115619]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:17.345 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129115]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:17.345 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:17.345 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129115]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:17.345 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129115]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:17.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:17.363+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbc000967f0 con 0x7fbc040ff730 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]: dispatch 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]': finished 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:17.607 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3442372583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115622]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115622]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:17.608 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115622]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:17.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129120]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129120]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129120]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 sudo[129120]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]: dispatch 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]': finished 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3442372583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:17.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:17 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]: dispatch 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/988516167' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a30f31c-10a5-4538-8fd9-35acf14be3eb"}]': finished 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3442372583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:17.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:17.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:17 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115625]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:18.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:18.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115625]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:18.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:17 smithi186 sudo[115625]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:18.339 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113710]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:50:18.339 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113710]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:18.339 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113710]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:18.339 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113710]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:18.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:18 smithi097 ceph-mon[99491]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113713]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:50:18.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113713]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:50:18.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113713]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:50:18.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 sudo[113713]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:50:18.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:18 smithi178 ceph-mon[103156]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:50:18.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:50:18.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:50:18.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:18 smithi186 ceph-mon[104393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:50:21.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:21 smithi097 ceph-mon[99491]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:21 smithi178 ceph-mon[103156]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:21 smithi186 ceph-mon[104393]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:23.382 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:23 smithi097 ceph-mon[99491]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:23.382 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:23 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:50:23.382 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:23 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:23 smithi178 ceph-mon[103156]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:23 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:50:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:23 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:23 smithi186 ceph-mon[104393]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:50:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:23 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:50:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:23 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:24.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:24 smithi097 ceph-mon[99491]: Deploying daemon osd.3 on smithi097 2023-10-08T09:50:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:24 smithi178 ceph-mon[103156]: Deploying daemon osd.3 on smithi097 2023-10-08T09:50:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:24 smithi186 ceph-mon[104393]: Deploying daemon osd.3 on smithi097 2023-10-08T09:50:25.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:25 smithi097 ceph-mon[99491]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:50:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:25 smithi178 ceph-mon[103156]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:50:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:25 smithi186 ceph-mon[104393]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:50:27.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:27 smithi097 ceph-mon[99491]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:50:27.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:27.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:27.418 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:27 smithi178 ceph-mon[103156]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:50:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:27 smithi186 ceph-mon[104393]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:50:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:29 smithi097 ceph-mon[99491]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:29 smithi178 ceph-mon[103156]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:29 smithi186 ceph-mon[104393]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:29.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.587+0000 7fbbfaffd700 1 -- 172.21.15.97:0/3507452400 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc0402cf80 con 0x7fbbf00a1470 2023-10-08T09:50:29.588 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 msgr2=0x7fbbf00a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 0x7fbbf00a3930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4006010 tx=0x7fbbf4009f10 comp rx=0 tx=0).stop 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 msgr2=0x7fbc04195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d9b0 tx=0x7fbc0000dd70 comp rx=0 tx=0).stop 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 shutdown_connections 2023-10-08T09:50:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbbf00a1470 0x7fbbf00a3930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc041012d0 0x7fbc0419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc040ff730 0x7fbc04195670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 --2- 172.21.15.97:0/3507452400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc04100930 0x7fbc04195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 >> 172.21.15.97:0/3507452400 conn(0x7fbc040facf0 msgr2=0x7fbc04103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 shutdown_connections 2023-10-08T09:50:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:50:29.589+0000 7fbc0c0da700 1 -- 172.21.15.97:0/3507452400 wait complete. 2023-10-08T09:50:30.016 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-08T09:50:30.019 INFO:tasks.cephadm:Deploying osd.4 on smithi178 with /dev/vg_nvme/lv_3... 2023-10-08T09:50:30.019 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:50:30.049 INFO:journalctl@ceph.osd.3.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:50:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:30.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:30 smithi178 ceph-mon[103156]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:50:30.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:30.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:30.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:30.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-mon[99491]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:30 smithi186 ceph-mon[104393]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:50:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:50:31.202 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 09:50:30 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T09:50:30.963+0000 7f3fc9c4c640 -1 osd.3 0 log_to_monitors true 2023-10-08T09:50:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:31 smithi097 ceph-mon[99491]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:50:31.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:31 smithi178 ceph-mon[103156]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:50:32.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:31 smithi186 ceph-mon[104393]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:50:32.058 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:50:32.062 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:50:32.062 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-08T09:50:32.062 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-08T09:50:32.062 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120677 s, 869 MB/s 2023-10-08T09:50:32.063 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-08T09:50:32.528 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2023-10-08T09:50:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:32 smithi178 ceph-mon[103156]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:50:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:32 smithi178 ceph-mon[103156]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:50:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:32 smithi178 ceph-mon[103156]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:50:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:32 smithi178 ceph-mon[103156]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:32.952 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T09:50:32.594+0000 7f3fbf2c3700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T09:50:32.953 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T09:50:32.601+0000 7f3fbb0b8700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:50:32.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-mon[99491]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:50:32.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-mon[99491]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:50:32.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-mon[99491]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:50:32.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:32.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:32 smithi097 ceph-mon[99491]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:32 smithi186 ceph-mon[104393]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:50:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:32 smithi186 ceph-mon[104393]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:50:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:32 smithi186 ceph-mon[104393]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:50:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:32 smithi186 ceph-mon[104393]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:50:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi097 2023-10-08T09:50:33.915 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi097 to 8584M 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:33.916 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi097 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi097 to 8584M 2023-10-08T09:50:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:33.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:33.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:33.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi097 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:34.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi097 to 8584M 2023-10-08T09:50:34.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:34.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:34.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:34.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:34.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.396+0000 7efd3457d700 1 -- 172.21.15.178:0/402883513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 msgr2=0x7efd2c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:34.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.396+0000 7efd3457d700 1 --2- 172.21.15.178:0/402883513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c107b80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efd2400b0d0 tx=0x7efd2400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 -- 172.21.15.178:0/402883513 shutdown_connections 2023-10-08T09:50:34.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 --2- 172.21.15.178:0/402883513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c107b80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 --2- 172.21.15.178:0/402883513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c0ff730 0x7efd2c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 --2- 172.21.15.178:0/402883513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c100930 0x7efd2c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 -- 172.21.15.178:0/402883513 >> 172.21.15.178:0/402883513 conn(0x7efd2c0facf0 msgr2=0x7efd2c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 -- 172.21.15.178:0/402883513 shutdown_connections 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 -- 172.21.15.178:0/402883513 wait complete. 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 Processor -- start 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.397+0000 7efd3457d700 1 -- start start 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c0ff730 0x7efd2c069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c100930 0x7efd2c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c19e2e0 con 0x7efd2c100930 2023-10-08T09:50:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c19e450 con 0x7efd2c1012d0 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c19e610 con 0x7efd2c0ff730 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45806/0 (socket says 172.21.15.178:45806) 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 -- 172.21.15.178:0/3918218502 learned_addr learned my addr 172.21.15.178:0/3918218502 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32319700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c0ff730 0x7efd2c069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd31b18700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c100930 0x7efd2c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 -- 172.21.15.178:0/3918218502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c0ff730 msgr2=0x7efd2c069570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c0ff730 0x7efd2c069570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 -- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c100930 msgr2=0x7efd2c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c100930 0x7efd2c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 -- 172.21.15.178:0/3918218502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009d00 con 0x7efd2c1012d0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd32b1a700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd24007bd0 tx=0x7efd24009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2400e040 con 0x7efd2c1012d0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2401dd50 con 0x7efd2c1012d0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c19e890 con 0x7efd2c1012d0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2401b450 con 0x7efd2c1012d0 2023-10-08T09:50:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.398+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c19ed00 con 0x7efd2c1012d0 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.399+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c106bc0 con 0x7efd2c1012d0 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.401+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efd2401d890 con 0x7efd2c1012d0 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.401+0000 7efd237fe700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 0x7efd180a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.401+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3702+0+0 (secure 0 0 0) 0x7efd240ce190 con 0x7efd2c1012d0 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.401+0000 7efd32319700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 0x7efd180a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.402+0000 7efd32319700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 0x7efd180a3940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd2c100790 tx=0x7efd1c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.405+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd24098a80 con 0x7efd2c1012d0 2023-10-08T09:50:34.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:34.604+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efd2c02cf80 con 0x7efd180a1480 2023-10-08T09:50:34.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] boot 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:34 smithi097 ceph-mon[99491]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] boot 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:34 smithi178 ceph-mon[103156]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: osd.3 [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] boot 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:34 smithi186 ceph-mon[104393]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:50:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:35 smithi178 ceph-mon[103156]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:35 smithi178 ceph-mon[103156]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:50:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:35 smithi097 ceph-mon[99491]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:35 smithi097 ceph-mon[99491]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:50:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:35 smithi186 ceph-mon[104393]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:35 smithi186 ceph-mon[104393]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:50:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:36 smithi097 ceph-mon[99491]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:36 smithi178 ceph-mon[103156]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:36 smithi186 ceph-mon[104393]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/3201856639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]': finished 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:50:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/3201856639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]': finished 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:50:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/3201856639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]: dispatch 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3cbf7e-d161-4d7f-959e-5943a5a63bd6"}]': finished 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:50:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/1225940676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:38 smithi097 ceph-mon[99491]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:50:38.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/1225940676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:38.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:38 smithi178 ceph-mon[103156]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:50:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/1225940676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:39.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:38 smithi186 ceph-mon[104393]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:50:41.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:41 smithi097 ceph-mon[99491]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-10-08T09:50:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:41 smithi178 ceph-mon[103156]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-10-08T09:50:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:41 smithi186 ceph-mon[104393]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-10-08T09:50:43.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:43 smithi178 ceph-mon[103156]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-10-08T09:50:43.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:43 smithi097 ceph-mon[99491]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-10-08T09:50:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:43 smithi186 ceph-mon[104393]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-10-08T09:50:44.394 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:44 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:50:44.394 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:44 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:44.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:44 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:50:44.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:44 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:44 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:50:44.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:44 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:45.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:45 smithi097 ceph-mon[99491]: Deploying daemon osd.4 on smithi178 2023-10-08T09:50:45.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:45 smithi097 ceph-mon[99491]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:50:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:45 smithi178 ceph-mon[103156]: Deploying daemon osd.4 on smithi178 2023-10-08T09:50:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:45 smithi178 ceph-mon[103156]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:50:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:45 smithi186 ceph-mon[104393]: Deploying daemon osd.4 on smithi178 2023-10-08T09:50:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:45 smithi186 ceph-mon[104393]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:50:47.388 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:47 smithi178 ceph-mon[103156]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-08T09:50:47.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:47 smithi097 ceph-mon[99491]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-08T09:50:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:47 smithi186 ceph-mon[104393]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-08T09:50:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:49 smithi178 ceph-mon[103156]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-08T09:50:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:49 smithi186 ceph-mon[104393]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-08T09:50:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:49.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:49 smithi097 ceph-mon[99491]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-10-08T09:50:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:51.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.013+0000 7efd237fe700 1 -- 172.21.15.178:0/3918218502 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd2c02cf80 con 0x7efd180a1480 2023-10-08T09:50:51.014 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 4 on host 'smithi178' 2023-10-08T09:50:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 msgr2=0x7efd180a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 0x7efd180a3940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd2c100790 tx=0x7efd1c00a380 comp rx=0 tx=0).stop 2023-10-08T09:50:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 msgr2=0x7efd2c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd24007bd0 tx=0x7efd24009440 comp rx=0 tx=0).stop 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 shutdown_connections 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd180a1480 0x7efd180a3940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd2c1012d0 0x7efd2c19db40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd2c0ff730 0x7efd2c069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 --2- 172.21.15.178:0/3918218502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd2c100930 0x7efd2c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 >> 172.21.15.178:0/3918218502 conn(0x7efd2c0facf0 msgr2=0x7efd2c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 shutdown_connections 2023-10-08T09:50:51.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:50:51.016+0000 7efd3457d700 1 -- 172.21.15.178:0/3918218502 wait complete. 2023-10-08T09:50:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:51.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:51.469 DEBUG:teuthology.orchestra.run.smithi178:osd.4> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-08T09:50:51.471 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2023-10-08T09:50:51.471 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:50:51.502 INFO:journalctl@ceph.osd.4.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:51.503 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.504 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:50:52.040 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:50:52.205 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 09:50:52 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T09:50:52.136+0000 7f097e370640 -1 osd.4 0 log_to_monitors true 2023-10-08T09:50:52.501 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:52 smithi178 ceph-mon[103156]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:52.501 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:52 smithi178 ceph-mon[103156]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:52.501 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:52 smithi178 ceph-mon[103156]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:52.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:52 smithi186 ceph-mon[104393]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:52.505 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:52 smithi186 ceph-mon[104393]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:52.506 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:52 smithi186 ceph-mon[104393]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:52.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:52 smithi097 ceph-mon[99491]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:52.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:52 smithi097 ceph-mon[99491]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:52.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:52 smithi097 ceph-mon[99491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111308 s, 942 MB/s 2023-10-08T09:50:53.393 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-08T09:50:53.513 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:53 smithi186 ceph-mon[104393]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:50:53.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:53 smithi186 ceph-mon[104393]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:53.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:53 smithi186 ceph-mon[104393]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:50:53.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:53 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:53.514 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:53 smithi186 ceph-mon[104393]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:53.674 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T09:50:53.221+0000 7f09739e7700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T09:50:53.674 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T09:50:53.227+0000 7f096f7dc700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:50:53.674 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-mon[103156]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:50:53.674 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-mon[103156]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:53.674 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-mon[103156]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:50:53.674 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:53.674 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:53 smithi178 ceph-mon[103156]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:53.694 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2023-10-08T09:50:53.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:53 smithi097 ceph-mon[99491]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:50:53.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:53 smithi097 ceph-mon[99491]: from='osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:53.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:53 smithi097 ceph-mon[99491]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:50:53.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:53 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:53.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:53 smithi097 ceph-mon[99491]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:50:54.295 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:50:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:50:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:50:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi178 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi178 to 8584M 2023-10-08T09:50:54.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:54.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:54.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:54.479 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi178 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi178 to 8584M 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.480 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:54.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:54.481 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi178 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:50:54.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi178 to 8584M 2023-10-08T09:50:54.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:54.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:54.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:50:54.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:50:55.328 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- 172.21.15.186:0/2363970231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc101ad0 msgr2=0x7f60cc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:55.328 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 --2- 172.21.15.186:0/2363970231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc101ad0 0x7f60cc101f30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009990 tx=0x7f60b8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- 172.21.15.186:0/2363970231 shutdown_connections 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 --2- 172.21.15.186:0/2363970231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc102470 0x7f60cc108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 --2- 172.21.15.186:0/2363970231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 0x7f60cc0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 --2- 172.21.15.186:0/2363970231 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc101ad0 0x7f60cc101f30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- 172.21.15.186:0/2363970231 >> 172.21.15.186:0/2363970231 conn(0x7f60cc0fa450 msgr2=0x7f60cc0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- 172.21.15.186:0/2363970231 shutdown_connections 2023-10-08T09:50:55.329 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- 172.21.15.186:0/2363970231 wait complete. 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 Processor -- start 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.328+0000 7f60d2afd700 1 -- start start 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 0x7f60cc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc101ad0 0x7f60cc196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc19e240 con 0x7f60cc101ad0 2023-10-08T09:50:55.331 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc19e380 con 0x7f60cc0feaf0 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc19e510 con 0x7f60cc102470 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:56988/0 (socket says 172.21.15.186:56988) 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 -- 172.21.15.186:0/3217293230 learned_addr learned my addr 172.21.15.186:0/3217293230 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d0899700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 0x7f60cc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60cbfff700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc101ad0 0x7f60cc196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 -- 172.21.15.186:0/3217293230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 msgr2=0x7f60cc196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 0x7f60cc196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 -- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc101ad0 msgr2=0x7f60cc196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc101ad0 0x7f60cc196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:50:55.332 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 -- 172.21.15.186:0/3217293230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80095f0 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d109a700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f60c400ef60 tx=0x7f60c400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4018070 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c400f040 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.329+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc19e790 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.330+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c400ccc0 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.330+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc19ec40 con 0x7f60cc102470 2023-10-08T09:50:55.333 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.331+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc107da0 con 0x7f60cc102470 2023-10-08T09:50:55.334 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.331+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f60c4003a00 con 0x7f60cc102470 2023-10-08T09:50:55.334 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.332+0000 7f60c9ffb700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 0x7f60bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:50:55.334 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.332+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f60c40cb840 con 0x7f60cc102470 2023-10-08T09:50:55.334 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.332+0000 7f60d0899700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 0x7f60bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:50:55.334 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.333+0000 7f60d0899700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 0x7f60bc0a39c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60cc067f90 tx=0x7f60c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:50:55.337 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.337+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f60c4095f00 con 0x7f60cc102470 2023-10-08T09:50:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:55 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:50:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:55 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:50:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:55 smithi186 ceph-mon[104393]: osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] boot 2023-10-08T09:50:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:55 smithi186 ceph-mon[104393]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:50:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:55.537 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:50:55.536+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f60cc060c30 con 0x7f60bc0a1500 2023-10-08T09:50:55.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:55 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:50:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:55 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:50:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:55 smithi097 ceph-mon[99491]: osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] boot 2023-10-08T09:50:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:55 smithi097 ceph-mon[99491]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:50:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:55 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:50:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:55 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:50:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:55 smithi178 ceph-mon[103156]: osd.4 [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] boot 2023-10-08T09:50:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:55 smithi178 ceph-mon[103156]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:50:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:56.488 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:56 smithi186 ceph-mon[104393]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:50:56.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:56 smithi097 ceph-mon[99491]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:56 smithi178 ceph-mon[103156]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:50:58.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/596041017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:57 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:57 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]': finished 2023-10-08T09:50:58.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:57 smithi097 ceph-mon[99491]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:50:58.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:50:58.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/596041017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:57 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:57 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]': finished 2023-10-08T09:50:58.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:57 smithi178 ceph-mon[103156]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:50:58.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:50:58.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/596041017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:57 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]: dispatch 2023-10-08T09:50:58.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:57 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7091360b-fa15-4e88-a5cc-44dcebef03c0"}]': finished 2023-10-08T09:50:58.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:57 smithi186 ceph-mon[104393]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:50:58.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:50:59.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:58 smithi097 ceph-mon[99491]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:59.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:50:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/2849781485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:59.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:58 smithi178 ceph-mon[103156]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:59.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:50:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/2849781485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:50:59.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:58 smithi186 ceph-mon[104393]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:50:59.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:50:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/2849781485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:01.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:01 smithi097 ceph-mon[99491]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:01 smithi178 ceph-mon[103156]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:01.483 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:01 smithi186 ceph-mon[104393]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:03 smithi097 ceph-mon[99491]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:03 smithi178 ceph-mon[103156]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:03 smithi186 ceph-mon[104393]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:04.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:04 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:51:04.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:04 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:04.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:04 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:51:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:04 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:04.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:04 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:51:04.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:04 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:05.377 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:05 smithi186 ceph-mon[104393]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:05.378 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:05 smithi186 ceph-mon[104393]: Deploying daemon osd.5 on smithi186 2023-10-08T09:51:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:05 smithi097 ceph-mon[99491]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:05.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:05 smithi097 ceph-mon[99491]: Deploying daemon osd.5 on smithi186 2023-10-08T09:51:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:05 smithi178 ceph-mon[103156]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:05 smithi178 ceph-mon[103156]: Deploying daemon osd.5 on smithi186 2023-10-08T09:51:07.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:07 smithi097 ceph-mon[99491]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:07 smithi178 ceph-mon[103156]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:07.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:07 smithi186 ceph-mon[104393]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:09 smithi186 ceph-mon[104393]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:09 smithi097 ceph-mon[99491]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:09 smithi178 ceph-mon[103156]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:10.650 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.649+0000 7f60c9ffb700 1 -- 172.21.15.186:0/3217293230 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f60cc060c30 con 0x7f60bc0a1500 2023-10-08T09:51:10.651 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2023-10-08T09:51:10.653 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 msgr2=0x7f60bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:10.653 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 0x7f60bc0a39c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60cc067f90 tx=0x7f60c000b410 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 msgr2=0x7f60cc19dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f60c400ef60 tx=0x7f60c400ef90 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 shutdown_connections 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f60bc0a1500 0x7f60bc0a39c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f60cc102470 0x7f60cc19dc70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60cc0feaf0 0x7f60cc196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 --2- 172.21.15.186:0/3217293230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60cc101ad0 0x7f60cc196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 >> 172.21.15.186:0/3217293230 conn(0x7f60cc0fa450 msgr2=0x7f60cc104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 shutdown_connections 2023-10-08T09:51:10.654 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:10.652+0000 7f60d2afd700 1 -- 172.21.15.186:0/3217293230 wait complete. 2023-10-08T09:51:11.083 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-08T09:51:11.085 INFO:tasks.cephadm:Deploying osd.6 on smithi097 with /dev/vg_nvme/lv_2... 2023-10-08T09:51:11.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:51:11.130 INFO:journalctl@ceph.osd.5.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:11.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:11.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:51:12.119 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 09:51:12 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T09:51:12.004+0000 7faaf078a640 -1 osd.5 0 log_to_monitors true 2023-10-08T09:51:12.371 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:12 smithi097 ceph-mon[99491]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:51:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:12 smithi178 ceph-mon[103156]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:51:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:12 smithi186 ceph-mon[104393]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-10-08T09:51:13.212 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-10-08T09:51:13.212 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0187248 s, 560 MB/s 2023-10-08T09:51:13.212 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-10-08T09:51:13.434 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T09:51:13.120+0000 7faae5e01700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T09:51:13.434 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T09:51:13.129+0000 7faadfbf2700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:51:13.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-mon[104393]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:13.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-mon[104393]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:51:13.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-mon[104393]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:51:13.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-mon[104393]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:51:13.435 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:13.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:13 smithi097 ceph-mon[99491]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:13.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:13 smithi097 ceph-mon[99491]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:51:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:13 smithi097 ceph-mon[99491]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:51:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:13 smithi097 ceph-mon[99491]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:51:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:13 smithi178 ceph-mon[103156]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:13 smithi178 ceph-mon[103156]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:51:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:13 smithi178 ceph-mon[103156]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:51:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:13 smithi178 ceph-mon[103156]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:51:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:13.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2023-10-08T09:51:14.182 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:51:14.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:51:14.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:51:14.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:14.385 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:51:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:51:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:14.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:51:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:51:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:14.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:15.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 -- 172.21.15.97:0/3179030856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c101320 msgr2=0x7fb81c103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:15.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 --2- 172.21.15.97:0/3179030856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c101320 0x7fb81c103710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00b0a0 tx=0x7fb80c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 -- 172.21.15.97:0/3179030856 shutdown_connections 2023-10-08T09:51:15.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 --2- 172.21.15.97:0/3179030856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 --2- 172.21.15.97:0/3179030856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c101320 0x7fb81c103710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 --2- 172.21.15.97:0/3179030856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c103c50 0x7fb81c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 -- 172.21.15.97:0/3179030856 >> 172.21.15.97:0/3179030856 conn(0x7fb81c0facf0 msgr2=0x7fb81c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 -- 172.21.15.97:0/3179030856 shutdown_connections 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.365+0000 7fb824083700 1 -- 172.21.15.97:0/3179030856 wait complete. 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 Processor -- start 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 -- start start 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c101320 0x7fb81c100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0fffb0 con 0x7fb81c103c50 2023-10-08T09:51:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c100120 con 0x7fb81c101320 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb824083700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a0970 con 0x7fb81c106580 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb82161e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb82161e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54418/0 (socket says 172.21.15.97:54418) 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb82161e700 1 -- 172.21.15.97:0/1513163439 learned_addr learned my addr 172.21.15.97:0/1513163439 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb821e1f700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c101320 0x7fb81c100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.366+0000 7fb822620700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 -- 172.21.15.97:0/1513163439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 msgr2=0x7fb81c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 -- 172.21.15.97:0/1513163439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c101320 msgr2=0x7fb81c100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:15.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c101320 0x7fb81c100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 -- 172.21.15.97:0/1513163439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c009d00 con 0x7fb81c103c50 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb822620700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb82161e700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c920 tx=0x7fb81000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810007b00 con 0x7fb81c103c50 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81000ce70 con 0x7fb81c103c50 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100054e0 con 0x7fb81c103c50 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c1a0c00 con 0x7fb81c103c50 2023-10-08T09:51:15.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.367+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c1a1100 con 0x7fb81c103c50 2023-10-08T09:51:15.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.368+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c0fc870 con 0x7fb81c103c50 2023-10-08T09:51:15.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.369+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb810004750 con 0x7fb81c103c50 2023-10-08T09:51:15.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.369+0000 7fb81affd700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 0x7fb8040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:15.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.369+0000 7fb821e1f700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 0x7fb8040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:15.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.370+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fb8100cb970 con 0x7fb81c103c50 2023-10-08T09:51:15.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.370+0000 7fb821e1f700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 0x7fb8040a39c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009fd0 tx=0x7fb80c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:15.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.374+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb810095e70 con 0x7fb81c103c50 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi186 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] boot 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:15 smithi097 ceph-mon[99491]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi186 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] boot 2023-10-08T09:51:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:51:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:15 smithi178 ceph-mon[103156]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi186 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:51:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: osd.5 [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] boot 2023-10-08T09:51:15.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:51:15.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:51:15.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:15 smithi186 ceph-mon[104393]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:51:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:15.572+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb81c02d020 con 0x7fb8040a1500 2023-10-08T09:51:16.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:16.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:16.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:16.390 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:16 smithi097 ceph-mon[99491]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:51:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:16 smithi178 ceph-mon[103156]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:51:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:16 smithi186 ceph-mon[104393]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:51:17.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:17 smithi097 ceph-mon[99491]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:17.162 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:17 smithi097 ceph-mon[99491]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:17 smithi178 ceph-mon[103156]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:17 smithi178 ceph-mon[103156]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:17 smithi186 ceph-mon[104393]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:17 smithi186 ceph-mon[104393]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:18.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:18 smithi097 ceph-mon[99491]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:51:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]: dispatch 2023-10-08T09:51:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]': finished 2023-10-08T09:51:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:18 smithi097 ceph-mon[99491]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:51:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:18 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:18 smithi178 ceph-mon[103156]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:51:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]: dispatch 2023-10-08T09:51:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]': finished 2023-10-08T09:51:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:18 smithi178 ceph-mon[103156]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:51:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:18 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:18 smithi186 ceph-mon[104393]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:51:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]: dispatch 2023-10-08T09:51:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1874673341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "facc86e5-e587-42d9-bbae-6f98916a084d"}]': finished 2023-10-08T09:51:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:18 smithi186 ceph-mon[104393]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:51:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:18 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:19 smithi097 ceph-mon[99491]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:51:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:19 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3693415934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:19 smithi178 ceph-mon[103156]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:51:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:19 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3693415934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:19 smithi186 ceph-mon[104393]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-08T09:51:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:19 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3693415934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:21 smithi097 ceph-mon[99491]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-08T09:51:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:21 smithi178 ceph-mon[103156]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-08T09:51:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:21 smithi186 ceph-mon[104393]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-08T09:51:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:23 smithi097 ceph-mon[99491]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:51:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:23 smithi178 ceph-mon[103156]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:51:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:23 smithi186 ceph-mon[104393]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:51:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:25 smithi097 ceph-mon[99491]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-08T09:51:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:51:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:25 smithi178 ceph-mon[103156]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-08T09:51:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:51:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:25 smithi186 ceph-mon[104393]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-08T09:51:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:51:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:26.314 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:26 smithi097 ceph-mon[99491]: Deploying daemon osd.6 on smithi097 2023-10-08T09:51:26.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:26 smithi178 ceph-mon[103156]: Deploying daemon osd.6 on smithi097 2023-10-08T09:51:26.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:26 smithi186 ceph-mon[104393]: Deploying daemon osd.6 on smithi097 2023-10-08T09:51:27.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:27 smithi097 ceph-mon[99491]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:51:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:27 smithi178 ceph-mon[103156]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:51:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:27 smithi186 ceph-mon[104393]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:51:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:29 smithi097 ceph-mon[99491]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:29 smithi178 ceph-mon[103156]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:29 smithi186 ceph-mon[104393]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:31.026 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 6 on host 'smithi097' 2023-10-08T09:51:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.025+0000 7fb81affd700 1 -- 172.21.15.97:0/1513163439 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb81c02d020 con 0x7fb8040a1500 2023-10-08T09:51:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 msgr2=0x7fb8040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 0x7fb8040a39c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009fd0 tx=0x7fb80c0060c0 comp rx=0 tx=0).stop 2023-10-08T09:51:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 msgr2=0x7fb81c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c920 tx=0x7fb81000cc30 comp rx=0 tx=0).stop 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 shutdown_connections 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8040a1500 0x7fb8040a39c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb81c106580 0x7fb81c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb81c101320 0x7fb81c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 --2- 172.21.15.97:0/1513163439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb81c103c50 0x7fb81c0ff2d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 >> 172.21.15.97:0/1513163439 conn(0x7fb81c0facf0 msgr2=0x7fb81c101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 shutdown_connections 2023-10-08T09:51:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:51:31.027+0000 7fb824083700 1 -- 172.21.15.97:0/1513163439 wait complete. 2023-10-08T09:51:31.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:31 smithi097 ceph-mon[99491]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:31.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:31.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:31 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:31 smithi178 ceph-mon[103156]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:31 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:31.460 DEBUG:teuthology.orchestra.run.smithi097:osd.6> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-08T09:51:31.462 INFO:tasks.cephadm:Deploying osd.7 on smithi178 with /dev/vg_nvme/lv_2... 2023-10-08T09:51:31.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:51:31.491 INFO:journalctl@ceph.osd.6.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:51:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:31 smithi186 ceph-mon[104393]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:31 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:32.041 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:51:32.703 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 09:51:32 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T09:51:32.451+0000 7fbe1063b640 -1 osd.6 0 log_to_monitors true 2023-10-08T09:51:32.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:32.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:32.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:32 smithi097 ceph-mon[99491]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:32.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:32 smithi097 ceph-mon[99491]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:51:32.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:32.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:32.804 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:32 smithi178 ceph-mon[103156]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:32.805 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:32 smithi178 ceph-mon[103156]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:51:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:32 smithi186 ceph-mon[104393]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:33.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:32 smithi186 ceph-mon[104393]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00807632 s, 1.3 GB/s 2023-10-08T09:51:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-08T09:51:33.882 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:33 smithi178 ceph-mon[103156]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:51:33.882 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:33 smithi178 ceph-mon[103156]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:51:33.882 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:33 smithi178 ceph-mon[103156]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:51:33.882 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:33 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:33.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2023-10-08T09:51:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-mon[99491]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:51:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-mon[99491]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:51:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-mon[99491]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:51:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:33.953 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T09:51:33.621+0000 7fbe05cb2700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T09:51:33.953 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 09:51:33 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T09:51:33.632+0000 7fbdfeaa1700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:51:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:33 smithi186 ceph-mon[104393]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:51:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:33 smithi186 ceph-mon[104393]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:51:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:33 smithi186 ceph-mon[104393]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:51:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:33 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:34.441 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:51:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:51:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:35.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:35.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 -- 172.21.15.178:0/4227549786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef940ff730 msgr2=0x7fef940ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:35.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 --2- 172.21.15.178:0/4227549786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef940ff730 0x7fef940ffb50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef840099c0 tx=0x7fef84009cd0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 -- 172.21.15.178:0/4227549786 shutdown_connections 2023-10-08T09:51:35.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 --2- 172.21.15.178:0/4227549786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef941012d0 0x7fef94107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 --2- 172.21.15.178:0/4227549786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef940ff730 0x7fef940ffb50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 --2- 172.21.15.178:0/4227549786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 0x7fef94100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 -- 172.21.15.178:0/4227549786 >> 172.21.15.178:0/4227549786 conn(0x7fef940facf0 msgr2=0x7fef940fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 -- 172.21.15.178:0/4227549786 shutdown_connections 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.639+0000 7fef9b357700 1 -- 172.21.15.178:0/4227549786 wait complete. 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 Processor -- start 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 -- start start 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef940ff730 0x7fef94068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 0x7fef9419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:35.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941a0410 con 0x7fef94100930 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941a0580 con 0x7fef941012d0 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef9b357700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941a0740 con 0x7fef940ff730 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45916/0 (socket says 172.21.15.178:45916) 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 -- 172.21.15.178:0/1840042405 learned_addr learned my addr 172.21.15.178:0/1840042405 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-08T09:51:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef988f2700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 0x7fef9419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef990f3700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef940ff730 0x7fef94068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 -- 172.21.15.178:0/1840042405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef940ff730 msgr2=0x7fef94068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef940ff730 0x7fef94068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 -- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 msgr2=0x7fef9419d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 0x7fef9419d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.640+0000 7fef998f4700 1 -- 172.21.15.178:0/1840042405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef941012d0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef998f4700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007ee0 tx=0x7fef8c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c017070 con 0x7fef941012d0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef941a0a20 con 0x7fef941012d0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8c00f040 con 0x7fef941012d0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c005740 con 0x7fef941012d0 2023-10-08T09:51:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.641+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef941a0f20 con 0x7fef941012d0 2023-10-08T09:51:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.642+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef94106c60 con 0x7fef941012d0 2023-10-08T09:51:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.643+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fef8c00dc60 con 0x7fef941012d0 2023-10-08T09:51:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.644+0000 7fef8a7fc700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 0x7fef800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.644+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fef8c0cbe60 con 0x7fef941012d0 2023-10-08T09:51:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.644+0000 7fef990f3700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 0x7fef800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.645+0000 7fef990f3700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 0x7fef800a3940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef84009990 tx=0x7fef8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.647+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef8c0961d0 con 0x7fef941012d0 2023-10-08T09:51:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:51:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:51:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi097 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi097 to 5722M 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] boot 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:35.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:35.846+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fef94060c30 con 0x7fef800a1480 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi097 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi097 to 5722M 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] boot 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:51:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:35.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:35.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:35.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:35.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:51:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:51:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi097 2023-10-08T09:51:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi097 to 5722M 2023-10-08T09:51:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: osd.6 [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] boot 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:36.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:36.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:36 smithi097 ceph-mon[99491]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:36 smithi178 ceph-mon[103156]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:36 smithi186 ceph-mon[104393]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/3647312790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]': finished 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:51:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/3647312790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]': finished 2023-10-08T09:51:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:51:39.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/3647312790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]: dispatch 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9764524a-05a3-4d06-83a8-c9c4ba6ec38c"}]': finished 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:51:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:40.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/2556283869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:40.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/2556283869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/2556283869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:51:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:41 smithi097 ceph-mon[99491]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:41 smithi178 ceph-mon[103156]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:41 smithi186 ceph-mon[104393]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:43.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:43 smithi097 ceph-mon[99491]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:43 smithi178 ceph-mon[103156]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:43 smithi186 ceph-mon[104393]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:45.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:45 smithi097 ceph-mon[99491]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:45.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:51:45.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:45 smithi178 ceph-mon[103156]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:51:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:45 smithi186 ceph-mon[104393]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:51:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:46.373 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:46 smithi178 ceph-mon[103156]: Deploying daemon osd.7 on smithi178 2023-10-08T09:51:46.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:46 smithi097 ceph-mon[99491]: Deploying daemon osd.7 on smithi178 2023-10-08T09:51:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:46 smithi186 ceph-mon[104393]: Deploying daemon osd.7 on smithi178 2023-10-08T09:51:47.414 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:47 smithi178 ceph-mon[103156]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:47.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:47 smithi097 ceph-mon[99491]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:47 smithi186 ceph-mon[104393]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:49.278 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:49 smithi178 ceph-mon[103156]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:49.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:49 smithi097 ceph-mon[99491]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:49 smithi186 ceph-mon[104393]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:50.442 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.442 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.443 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:50.443 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:50 smithi178 ceph-mon[103156]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:50.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:50.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:50 smithi097 ceph-mon[99491]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:50 smithi186 ceph-mon[104393]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.956+0000 7fef8a7fc700 1 -- 172.21.15.178:0/1840042405 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef94060c30 con 0x7fef800a1480 2023-10-08T09:51:51.957 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 7 on host 'smithi178' 2023-10-08T09:51:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.958+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 msgr2=0x7fef800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.958+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 0x7fef800a3940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef84009990 tx=0x7fef8401a040 comp rx=0 tx=0).stop 2023-10-08T09:51:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.958+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 msgr2=0x7fef9419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.958+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007ee0 tx=0x7fef8c00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:51:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.958+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 shutdown_connections 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef800a1480 0x7fef800a3940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef941012d0 0x7fef9419fdf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef940ff730 0x7fef94068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 --2- 172.21.15.178:0/1840042405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef94100930 0x7fef9419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 >> 172.21.15.178:0/1840042405 conn(0x7fef940facf0 msgr2=0x7fef94103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 shutdown_connections 2023-10-08T09:51:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:51:51.959+0000 7fef9b357700 1 -- 172.21.15.178:0/1840042405 wait complete. 2023-10-08T09:51:52.319 DEBUG:teuthology.orchestra.run.smithi178:osd.7> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-08T09:51:52.321 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2023-10-08T09:51:52.321 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:51:52.353 INFO:journalctl@ceph.osd.7.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:51:52.890 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:51:53.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:53.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:53.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:53.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.067 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:53.068 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.068 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.068 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:53.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:51:53.373 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 09:51:53 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T09:51:53.254+0000 7fe4336b1640 -1 osd.7 0 log_to_monitors true 2023-10-08T09:51:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:53 smithi186 ceph-mon[104393]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:53 smithi186 ceph-mon[104393]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:53 smithi097 ceph-mon[99491]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:53 smithi097 ceph-mon[99491]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:53 smithi178 ceph-mon[103156]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:53 smithi178 ceph-mon[103156]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129924 s, 807 MB/s 2023-10-08T09:51:54.260 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-08T09:51:54.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:54 smithi186 ceph-mon[104393]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:55.102 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:54 smithi097 ceph-mon[99491]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:55.205 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T09:51:54.757+0000 7fe428d28700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T09:51:55.205 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T09:51:54.768+0000 7fe421316700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:51:55.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:51:55.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: from='osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:51:55.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:55.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:51:55.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:54 smithi178 ceph-mon[103156]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi178 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:55.937 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi178 to 5722M 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] boot 2023-10-08T09:51:55.938 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:51:55.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.133+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2962096872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df4100050 msgr2=0x7f1df41004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:56.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.133+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2962096872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df4100050 0x7f1df41004d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009990 tx=0x7f1de4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.135 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2962096872 shutdown_connections 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2962096872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 0x7f1df4107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2962096872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df40fee60 0x7f1df40ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2962096872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df4100050 0x7f1df41004d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2962096872 >> 172.21.15.186:0/2962096872 conn(0x7f1df40fa450 msgr2=0x7f1df40fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2962096872 shutdown_connections 2023-10-08T09:51:56.136 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2962096872 wait complete. 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 Processor -- start 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.134+0000 7f1df8e0e700 1 -- start start 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 0x7f1df4195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 0x7f1df419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df419d1a0 con 0x7f1df4100050 2023-10-08T09:51:56.138 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df419d2e0 con 0x7f1df4100a10 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4107b70 con 0x7f1df40fee60 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57100/0 (socket says 172.21.15.186:57100) 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 -- 172.21.15.186:0/2161185164 learned_addr learned my addr 172.21.15.186:0/2161185164 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df1d9b700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 0x7f1df4195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df2d9d700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 0x7f1df419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 -- 172.21.15.186:0/2161185164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 msgr2=0x7f1df419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 0x7f1df419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 -- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 msgr2=0x7f1df4195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 0x7f1df4195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 -- 172.21.15.186:0/2161185164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de40095f0 con 0x7f1df40fee60 2023-10-08T09:51:56.139 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df1d9b700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 0x7f1df4195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df259c700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e970 tx=0x7f1ddc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc009790 con 0x7f1df40fee60 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ddc0044e0 con 0x7f1df40fee60 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df4107e50 con 0x7f1df40fee60 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc003e30 con 0x7f1df40fee60 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.135+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df4108350 con 0x7f1df40fee60 2023-10-08T09:51:56.140 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.136+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df4106360 con 0x7f1df40fee60 2023-10-08T09:51:56.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.137+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1ddc003f90 con 0x7f1df40fee60 2023-10-08T09:51:56.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.138+0000 7f1deb7fe700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 0x7f1de00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:51:56.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.138+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f1ddc014070 con 0x7f1df40fee60 2023-10-08T09:51:56.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.138+0000 7f1df1d9b700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 0x7f1de00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:51:56.141 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.139+0000 7f1df1d9b700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 0x7f1de00a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de400b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:51:56.143 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.142+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ddc096810 con 0x7f1df40fee60 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi178 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi178 to 5722M 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] boot 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:51:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi178 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi178 to 5722M 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: osd.7 [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] boot 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:51:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:51:56.341 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:51:56.340+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1df41085d0 con 0x7f1de00a1480 2023-10-08T09:51:57.421 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:51:57.421 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:51:57.421 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:57.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:51:57.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:57.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:57.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:57.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:51:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:51:58.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:58 smithi097 ceph-mon[99491]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:58.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:58 smithi097 ceph-mon[99491]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:51:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:58 smithi178 ceph-mon[103156]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:58 smithi178 ceph-mon[103156]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:51:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:58 smithi186 ceph-mon[104393]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:51:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:58 smithi186 ceph-mon[104393]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/2610281363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]': finished 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:51:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:51:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/2610281363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]': finished 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:51:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:51:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:51:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: pgmap v167: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:51:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:51:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/2610281363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]: dispatch 2023-10-08T09:51:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4415829-318a-4bb8-8e77-feda54434e5f"}]': finished 2023-10-08T09:51:59.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:51:59.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:51:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:00.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/3747819892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/3747819892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/3747819892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:01 smithi097 ceph-mon[99491]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:01 smithi178 ceph-mon[103156]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:01 smithi186 ceph-mon[104393]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:03.399 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:03 smithi186 ceph-mon[104393]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:03.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:03 smithi097 ceph-mon[99491]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:03 smithi178 ceph-mon[103156]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:05.427 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:05 smithi186 ceph-mon[104393]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:05.427 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:05 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:52:05.427 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:05 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:05 smithi097 ceph-mon[99491]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:05 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:52:05.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:05 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:05 smithi178 ceph-mon[103156]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:05 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:52:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:05 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:06.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:06 smithi097 ceph-mon[99491]: Deploying daemon osd.8 on smithi186 2023-10-08T09:52:06.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:06 smithi178 ceph-mon[103156]: Deploying daemon osd.8 on smithi186 2023-10-08T09:52:06.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:06 smithi186 ceph-mon[104393]: Deploying daemon osd.8 on smithi186 2023-10-08T09:52:07.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:07 smithi097 ceph-mon[99491]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:07 smithi178 ceph-mon[103156]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:07.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:07 smithi186 ceph-mon[104393]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:09.431 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:09 smithi186 ceph-mon[104393]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:09.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.432 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:09 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:09.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:09 smithi097 ceph-mon[99491]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:09.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:09 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:09 smithi178 ceph-mon[103156]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:09 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:11.346 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:11 smithi186 ceph-mon[104393]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:11.447 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.446+0000 7f1deb7fe700 1 -- 172.21.15.186:0/2161185164 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1df41085d0 con 0x7f1de00a1480 2023-10-08T09:52:11.447 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2023-10-08T09:52:11.449 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 msgr2=0x7f1de00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:11.449 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 0x7f1de00a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de400b770 comp rx=0 tx=0).stop 2023-10-08T09:52:11.449 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 msgr2=0x7f1df41956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:11.449 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e970 tx=0x7f1ddc00ed30 comp rx=0 tx=0).stop 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 shutdown_connections 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1de00a1480 0x7f1de00a3940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df4100a10 0x7f1df419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1df40fee60 0x7f1df41956b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 --2- 172.21.15.186:0/2161185164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df4100050 0x7f1df4195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.447+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 >> 172.21.15.186:0/2161185164 conn(0x7f1df40fa450 msgr2=0x7f1df41034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.448+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 shutdown_connections 2023-10-08T09:52:11.450 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:11.448+0000 7f1df8e0e700 1 -- 172.21.15.186:0/2161185164 wait complete. 2023-10-08T09:52:11.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:11 smithi097 ceph-mon[99491]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:11 smithi178 ceph-mon[103156]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:11.780 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-08T09:52:11.782 INFO:tasks.cephadm:Deploying osd.9 on smithi097 with /dev/vg_nvme/lv_1... 2023-10-08T09:52:11.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:52:11.810 INFO:journalctl@ceph.osd.8.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:52:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:12.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:12 smithi097 ceph-mon[99491]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:12.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:12.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:12 smithi178 ceph-mon[103156]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:12.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:12.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:12.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.769 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.769 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:12.769 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:12.769 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-mon[104393]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:13.019 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 09:52:12 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T09:52:12.979+0000 7fddf5afc640 -1 osd.8 0 log_to_monitors true 2023-10-08T09:52:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:13 smithi097 ceph-mon[99491]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:13 smithi097 ceph-mon[99491]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:13 smithi178 ceph-mon[103156]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:13 smithi178 ceph-mon[103156]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:13 smithi186 ceph-mon[104393]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:13 smithi186 ceph-mon[104393]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:52:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:52:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:52:13.825 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-10-08T09:52:13.826 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-10-08T09:52:13.826 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112956 s, 928 MB/s 2023-10-08T09:52:13.826 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-10-08T09:52:14.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:14.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:14.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:52:14.775 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T09:52:14.410+0000 7fddeb173700 -1 osd.8 0 waiting for initial osdmap 2023-10-08T09:52:14.775 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T09:52:14.421+0000 7fdde6f68700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:52:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:14.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi186 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] boot 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:52:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi186 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] boot 2023-10-08T09:52:15.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:52:15.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi186 2023-10-08T09:52:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-08T09:52:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:52:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:52:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: osd.8 [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] boot 2023-10-08T09:52:15.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:52:15.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:52:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 -- 172.21.15.97:0/810151292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc100930 msgr2=0x7f4ecc100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/810151292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc100930 0x7f4ecc100d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099c0 tx=0x7f4eb4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 -- 172.21.15.97:0/810151292 shutdown_connections 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/810151292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/810151292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc0ff730 0x7f4ecc0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/810151292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc100930 0x7f4ecc100d90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 -- 172.21.15.97:0/810151292 >> 172.21.15.97:0/810151292 conn(0x7f4ecc0facf0 msgr2=0x7f4ecc0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 -- 172.21.15.97:0/810151292 shutdown_connections 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.977+0000 7f4ed2a56700 1 -- 172.21.15.97:0/810151292 wait complete. 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 Processor -- start 2023-10-08T09:52:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 -- start start 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc0ff730 0x7f4ecc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc100930 0x7f4ecc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1a04a0 con 0x7f4ecc1012d0 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1a05e0 con 0x7f4ecc100930 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.978+0000 7f4ed2a56700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1a0770 con 0x7f4ecc0ff730 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54560/0 (socket says 172.21.15.97:54560) 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 -- 172.21.15.97:0/3092178752 learned_addr learned my addr 172.21.15.97:0/3092178752 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ecbfff700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc0ff730 0x7f4ecc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ecb7fe700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc100930 0x7f4ecc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 -- 172.21.15.97:0/3092178752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc0ff730 msgr2=0x7f4ecc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc0ff730 0x7f4ecc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 -- 172.21.15.97:0/3092178752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc100930 msgr2=0x7f4ecc198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc100930 0x7f4ecc198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 -- 172.21.15.97:0/3092178752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40095f0 con 0x7f4ecc1012d0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed0ff3700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007f10 tx=0x7f4ec400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4017070 con 0x7f4ecc1012d0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc1a0a50 con 0x7f4ecc1012d0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec400f040 con 0x7f4ecc1012d0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4005740 con 0x7f4ecc1012d0 2023-10-08T09:52:15.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.979+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc1a0f50 con 0x7f4ecc1012d0 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.980+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc04e4c0 con 0x7f4ecc1012d0 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.982+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4ec40058a0 con 0x7f4ecc1012d0 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.982+0000 7f4ec97fa700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 0x7f4eb80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.983+0000 7f4ecbfff700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 0x7f4eb80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.983+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f4ec40cc1b0 con 0x7f4ecc1012d0 2023-10-08T09:52:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.983+0000 7f4ecbfff700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 0x7f4eb80a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc100790 tx=0x7f4ebc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:15.986+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ec40961a0 con 0x7f4ecc1012d0 2023-10-08T09:52:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:16.188+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ecc0fcd50 con 0x7f4eb80a1480 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:16.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:16.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:18.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:18 smithi097 ceph-mon[99491]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:52:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:18 smithi178 ceph-mon[103156]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:52:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:18 smithi186 ceph-mon[104393]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]: dispatch 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]': finished 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: osdmap e60: 10 total, 9 up, 10 in 2023-10-08T09:52:19.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:19 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]: dispatch 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]': finished 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: osdmap e60: 10 total, 9 up, 10 in 2023-10-08T09:52:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:19 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]: dispatch 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2177175231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91"}]': finished 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: osdmap e60: 10 total, 9 up, 10 in 2023-10-08T09:52:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:19 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:20.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:20 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2626795607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:20.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:20 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2626795607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:20.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:20 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2626795607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:21 smithi097 ceph-mon[99491]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:21 smithi178 ceph-mon[103156]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:21 smithi186 ceph-mon[104393]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:23 smithi097 ceph-mon[99491]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-08T09:52:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:23 smithi178 ceph-mon[103156]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-08T09:52:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:23 smithi186 ceph-mon[104393]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-08T09:52:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:25 smithi097 ceph-mon[99491]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-08T09:52:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:25 smithi178 ceph-mon[103156]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-08T09:52:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:25 smithi186 ceph-mon[104393]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-10-08T09:52:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:26 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:52:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:26 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:26 smithi097 ceph-mon[99491]: Deploying daemon osd.9 on smithi097 2023-10-08T09:52:26.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:26 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:52:26.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:26 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:26.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:26 smithi178 ceph-mon[103156]: Deploying daemon osd.9 on smithi097 2023-10-08T09:52:26.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:26 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:52:26.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:26 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:26.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:26 smithi186 ceph-mon[104393]: Deploying daemon osd.9 on smithi097 2023-10-08T09:52:27.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:27 smithi097 ceph-mon[99491]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:52:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:27 smithi178 ceph-mon[103156]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:52:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:27 smithi186 ceph-mon[104393]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:52:29.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:29 smithi097 ceph-mon[99491]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-08T09:52:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:29 smithi178 ceph-mon[103156]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-08T09:52:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:29 smithi186 ceph-mon[104393]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-08T09:52:30.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:30.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:30 smithi097 ceph-mon[99491]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:52:30.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:30.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:30 smithi178 ceph-mon[103156]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:52:30.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:30.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:30.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:30 smithi186 ceph-mon[104393]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:52:31.999 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 9 on host 'smithi097' 2023-10-08T09:52:31.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:31.998+0000 7f4ec97fa700 1 -- 172.21.15.97:0/3092178752 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4ecc0fcd50 con 0x7f4eb80a1480 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 msgr2=0x7f4eb80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 0x7f4eb80a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc100790 tx=0x7f4ebc00a380 comp rx=0 tx=0).stop 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 msgr2=0x7f4ecc19fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007f10 tx=0x7f4ec400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 shutdown_connections 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4eb80a1480 0x7f4eb80a3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ecc1012d0 0x7f4ecc19fed0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:32.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ecc0ff730 0x7f4ecc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:32.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 --2- 172.21.15.97:0/3092178752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ecc100930 0x7f4ecc198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:32.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 >> 172.21.15.97:0/3092178752 conn(0x7f4ecc0facf0 msgr2=0x7f4ecc0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:32.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 shutdown_connections 2023-10-08T09:52:32.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:52:32.000+0000 7f4ed2a56700 1 -- 172.21.15.97:0/3092178752 wait complete. 2023-10-08T09:52:32.449 DEBUG:teuthology.orchestra.run.smithi097:osd.9> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-08T09:52:32.451 INFO:tasks.cephadm:Deploying osd.10 on smithi178 with /dev/vg_nvme/lv_1... 2023-10-08T09:52:32.451 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:52:32.482 INFO:journalctl@ceph.osd.9.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:52:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:52:33.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:32 smithi097 ceph-mon[99491]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:52:33.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:32 smithi178 ceph-mon[103156]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:52:33.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:32 smithi186 ceph-mon[104393]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:52:33.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:33.703 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 09:52:33 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T09:52:33.343+0000 7fd93edaa640 -1 osd.9 0 log_to_monitors true 2023-10-08T09:52:34.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:33 smithi097 ceph-mon[99491]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:52:34.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:33 smithi178 ceph-mon[103156]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:52:34.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:33 smithi186 ceph-mon[104393]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00829304 s, 1.3 GB/s 2023-10-08T09:52:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-08T09:52:34.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2023-10-08T09:52:34.946 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-mon[99491]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:34.947 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-mon[99491]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:52:34.947 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-mon[99491]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:52:34.947 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-mon[99491]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:52:34.947 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:35.202 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T09:52:34.947+0000 7fd934421700 -1 osd.9 0 waiting for initial osdmap 2023-10-08T09:52:35.274 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 09:52:34 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T09:52:34.959+0000 7fd92f214700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:52:35.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:34 smithi186 ceph-mon[104393]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:35.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:34 smithi186 ceph-mon[104393]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:52:35.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:34 smithi186 ceph-mon[104393]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:52:35.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:34 smithi186 ceph-mon[104393]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:52:35.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:35.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:34 smithi178 ceph-mon[103156]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:35.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:34 smithi178 ceph-mon[103156]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:52:35.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:34 smithi178 ceph-mon[103156]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:52:35.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:34 smithi178 ceph-mon[103156]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-10-08T09:52:35.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:35.435 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:52:36.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:52:36.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:52:36.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi097 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi097 to 4292M 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:36.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.207 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:35 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi097 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi097 to 4292M 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:35 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi097 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi097 to 4292M 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:36.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:35 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:36.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 -- 172.21.15.178:0/3619987631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc214100950 msgr2=0x7fc214100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 --2- 172.21.15.178:0/3619987631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc214100950 0x7fc214100db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2040099c0 tx=0x7fc204009cd0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 -- 172.21.15.178:0/3619987631 shutdown_connections 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 --2- 172.21.15.178:0/3619987631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2141012f0 0x7fc214107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 --2- 172.21.15.178:0/3619987631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 0x7fc2140ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 --2- 172.21.15.178:0/3619987631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc214100950 0x7fc214100db0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 -- 172.21.15.178:0/3619987631 >> 172.21.15.178:0/3619987631 conn(0x7fc2140facf0 msgr2=0x7fc2140fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 -- 172.21.15.178:0/3619987631 shutdown_connections 2023-10-08T09:52:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.629+0000 7fc218f7b700 1 -- 172.21.15.178:0/3619987631 wait complete. 2023-10-08T09:52:36.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 Processor -- start 2023-10-08T09:52:36.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 -- start start 2023-10-08T09:52:36.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 0x7fc214103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:36.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 0x7fc214101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:36.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214102a10 con 0x7fc2140ff750 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214102b80 con 0x7fc2141012f0 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc218f7b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2141a07a0 con 0x7fc214100950 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:46058/0 (socket says 172.21.15.178:46058) 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 -- 172.21.15.178:0/4288469204 learned_addr learned my addr 172.21.15.178:0/4288469204 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc21259c700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 0x7fc214103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc211d9b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 0x7fc214101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 -- 172.21.15.178:0/4288469204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 msgr2=0x7fc214101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 0x7fc214101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 -- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 msgr2=0x7fc214103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:36.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 0x7fc214103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 -- 172.21.15.178:0/4288469204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040095f0 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc211d9b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 0x7fc214101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.630+0000 7fc212d9d700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00ada0 tx=0x7fc20c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.631+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00f070 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.631+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc20c007b30 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.631+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2141a0a30 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.631+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0053e0 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.631+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141a0ee0 con 0x7fc2141012f0 2023-10-08T09:52:36.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.632+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21404e4c0 con 0x7fc2141012f0 2023-10-08T09:52:36.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.633+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc20c00c980 con 0x7fc2141012f0 2023-10-08T09:52:36.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.633+0000 7fc2037fe700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 0x7fc1fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:36.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.633+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc20c0cbb10 con 0x7fc2141012f0 2023-10-08T09:52:36.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.634+0000 7fc21259c700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 0x7fc1fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:36.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.634+0000 7fc21259c700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 0x7fc1fc0a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2141007b0 tx=0x7fc208009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:36.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.638+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc20c095a00 con 0x7fc2141012f0 2023-10-08T09:52:36.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:36.833+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2140695a0 con 0x7fc1fc0a1480 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] boot 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: osdmap e63: 10 total, 10 up, 10 in 2023-10-08T09:52:37.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:37.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:37.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:37.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] boot 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: osdmap e63: 10 total, 10 up, 10 in 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:37.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:37.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:37.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:52:37.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:52:37.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:52:37.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: osd.9 [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] boot 2023-10-08T09:52:37.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: osdmap e63: 10 total, 10 up, 10 in 2023-10-08T09:52:37.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:52:37.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:37.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:37.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:38.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:37 smithi178 ceph-mon[103156]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:38.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:37 smithi178 ceph-mon[103156]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:52:38.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:37 smithi186 ceph-mon[104393]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:38.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:37 smithi186 ceph-mon[104393]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:52:38.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:37 smithi097 ceph-mon[99491]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:38.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:37 smithi097 ceph-mon[99491]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:52:39.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:38 smithi186 ceph-mon[104393]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:39.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:38 smithi097 ceph-mon[99491]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:39.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:38 smithi178 ceph-mon[103156]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/121390546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]': finished 2023-10-08T09:52:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: osdmap e65: 11 total, 10 up, 11 in 2023-10-08T09:52:40.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:40.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.178:0/3635797220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/121390546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]': finished 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: osdmap e65: 11 total, 10 up, 11 in 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:40.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.178:0/3635797220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/121390546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]: dispatch 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0e6cc7-619d-466b-9ede-4bfc9d404bba"}]': finished 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: osdmap e65: 11 total, 10 up, 11 in 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:40.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.178:0/3635797220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:52:41.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:41 smithi097 ceph-mon[99491]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:41.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:41 smithi178 ceph-mon[103156]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:41 smithi186 ceph-mon[104393]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:43.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:43 smithi097 ceph-mon[99491]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:43 smithi178 ceph-mon[103156]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:43 smithi186 ceph-mon[104393]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:45.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:45 smithi097 ceph-mon[99491]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:45 smithi178 ceph-mon[103156]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:45 smithi186 ceph-mon[104393]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:46.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:52:46.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:46.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:52:46.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:52:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:47.350 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:47 smithi178 ceph-mon[103156]: Deploying daemon osd.10 on smithi178 2023-10-08T09:52:47.350 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:47 smithi178 ceph-mon[103156]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:47.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:47 smithi097 ceph-mon[99491]: Deploying daemon osd.10 on smithi178 2023-10-08T09:52:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:47 smithi097 ceph-mon[99491]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:47 smithi186 ceph-mon[104393]: Deploying daemon osd.10 on smithi178 2023-10-08T09:52:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:47 smithi186 ceph-mon[104393]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:49.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:49 smithi178 ceph-mon[103156]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:49.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:49 smithi097 ceph-mon[99491]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:49 smithi186 ceph-mon[104393]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:50.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:50 smithi178 ceph-mon[103156]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:51.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:51.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:50 smithi097 ceph-mon[99491]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:51.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:51.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:51.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:51.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:50 smithi186 ceph-mon[104393]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:52.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.389+0000 7fc2037fe700 1 -- 172.21.15.178:0/4288469204 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc2140695a0 con 0x7fc1fc0a1480 2023-10-08T09:52:52.390 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 10 on host 'smithi178' 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.391+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 msgr2=0x7fc1fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.391+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 0x7fc1fc0a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2141007b0 tx=0x7fc208009260 comp rx=0 tx=0).stop 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 msgr2=0x7fc214102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00ada0 tx=0x7fc20c00c330 comp rx=0 tx=0).stop 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 shutdown_connections 2023-10-08T09:52:52.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc1fc0a1480 0x7fc1fc0a3940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2141012f0 0x7fc214102270 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2140ff750 0x7fc214103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 --2- 172.21.15.178:0/4288469204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc214100950 0x7fc214101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 >> 172.21.15.178:0/4288469204 conn(0x7fc2140facf0 msgr2=0x7fc214068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 shutdown_connections 2023-10-08T09:52:52.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-08T09:52:52.392+0000 7fc218f7b700 1 -- 172.21.15.178:0/4288469204 wait complete. 2023-10-08T09:52:52.824 DEBUG:teuthology.orchestra.run.smithi178:osd.10> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-08T09:52:52.826 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2023-10-08T09:52:52.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:52:52.855 INFO:journalctl@ceph.osd.10.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:52:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:52 smithi097 ceph-mon[99491]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:52 smithi178 ceph-mon[103156]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:53.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:52 smithi186 ceph-mon[104393]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:53.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:53.398 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:52:54.187 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:53 smithi178 ceph-mon[103156]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.187 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:53 smithi178 ceph-mon[103156]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.187 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 09:52:53 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T09:52:53.795+0000 7f40d8502640 -1 osd.10 0 log_to_monitors true 2023-10-08T09:52:54.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:53 smithi097 ceph-mon[99491]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:53 smithi097 ceph-mon[99491]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.269 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:53 smithi186 ceph-mon[104393]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.269 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:53 smithi186 ceph-mon[104393]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01124 s, 933 MB/s 2023-10-08T09:52:54.756 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.067 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:54 smithi186 ceph-mon[104393]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.147 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2023-10-08T09:52:55.190 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T09:52:54.929+0000 7f40cdb79700 -1 osd.10 0 waiting for initial osdmap 2023-10-08T09:52:55.190 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T09:52:54.943+0000 7f40c7169700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:52:55.190 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:55.190 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:52:55.190 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:52:55.190 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:55.190 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.191 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:54 smithi178 ceph-mon[103156]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: from='osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:54 smithi097 ceph-mon[99491]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-08T09:52:55.705 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:52:56.162 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] boot 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: osdmap e68: 11 total, 11 up, 11 in 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.163 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.164 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] boot 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: osdmap e68: 11 total, 11 up, 11 in 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: osd.10 [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] boot 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: osdmap e68: 11 total, 11 up, 11 in 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:52:56.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:52:56.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 -- 172.21.15.186:0/689186380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 msgr2=0x7f605c0ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:56.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 --2- 172.21.15.186:0/689186380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c0ffb70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6050009990 tx=0x7f6050009ca0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 -- 172.21.15.186:0/689186380 shutdown_connections 2023-10-08T09:52:56.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 --2- 172.21.15.186:0/689186380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f605c1012f0 0x7f605c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.782 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 --2- 172.21.15.186:0/689186380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c0ffb70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 --2- 172.21.15.186:0/689186380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 -- 172.21.15.186:0/689186380 >> 172.21.15.186:0/689186380 conn(0x7f605c0facf0 msgr2=0x7f605c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 -- 172.21.15.186:0/689186380 shutdown_connections 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.778+0000 7f6062d8c700 1 -- 172.21.15.186:0/689186380 wait complete. 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 Processor -- start 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 -- start start 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f605c1012f0 0x7f605c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:56.783 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c19e230 con 0x7f605c100950 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c19e3a0 con 0x7f605c1012f0 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6062d8c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c19e560 con 0x7f605c0ff750 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57248/0 (socket says 172.21.15.186:57248) 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 -- 172.21.15.186:0/3104214757 learned_addr learned my addr 172.21.15.186:0/3104214757 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f605bfff700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 -- 172.21.15.186:0/3104214757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f605c1012f0 msgr2=0x7f605c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f605c1012f0 0x7f605c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 -- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 msgr2=0x7f605c19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:52:56.784 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 -- 172.21.15.186:0/3104214757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500095f0 con 0x7f605c0ff750 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f605bfff700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6060b28700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f605000f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.779+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605001d070 con 0x7f605c0ff750 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.780+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605000fd00 con 0x7f605c0ff750 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.780+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c19e7e0 con 0x7f605c0ff750 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.780+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60500176c0 con 0x7f605c0ff750 2023-10-08T09:52:56.785 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.780+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c19ebf0 con 0x7f605c0ff750 2023-10-08T09:52:56.786 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.781+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c106c00 con 0x7f605c0ff750 2023-10-08T09:52:56.786 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.782+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6050022470 con 0x7f605c0ff750 2023-10-08T09:52:56.786 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.782+0000 7f6059ffb700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 0x7f604c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:52:56.786 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.783+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f60500cdb20 con 0x7f605c0ff750 2023-10-08T09:52:56.786 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.783+0000 7f605bfff700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 0x7f604c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:52:56.787 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.783+0000 7f605bfff700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 0x7f604c0a3940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6048005fd0 tx=0x7f6048005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:52:56.788 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.787+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f605005d100 con 0x7f605c0ff750 2023-10-08T09:52:56.986 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:52:56.985+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f605c02cfd0 con 0x7f604c0a1480 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi178 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi178 to 4292M 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:57.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:57 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi178 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi178 to 4292M 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:57.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:57 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi178 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi178 to 4292M 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:52:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:52:57.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:57 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:52:58.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:58 smithi097 ceph-mon[99491]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:58.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:58 smithi097 ceph-mon[99491]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:52:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:58 smithi178 ceph-mon[103156]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:58 smithi178 ceph-mon[103156]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:52:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:58 smithi186 ceph-mon[104393]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:52:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:58 smithi186 ceph-mon[104393]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:52:59.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:59 smithi097 ceph-mon[99491]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:52:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:52:59 smithi097 ceph-mon[99491]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:52:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:59 smithi178 ceph-mon[103156]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:52:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:52:59 smithi178 ceph-mon[103156]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:52:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:59 smithi186 ceph-mon[104393]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:52:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:52:59 smithi186 ceph-mon[104393]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:53:00.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/1589743677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]': finished 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:00.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.186:0/1075067180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/1589743677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]': finished 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.186:0/1075067180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/1589743677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]: dispatch 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34036837-0a5c-4d09-b8cb-ff074305f8b5"}]': finished 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:53:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:00.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.186:0/1075067180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:53:01.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:01 smithi097 ceph-mon[99491]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:01 smithi178 ceph-mon[103156]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:01 smithi186 ceph-mon[104393]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:03.351 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:03 smithi186 ceph-mon[104393]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-08T09:53:03.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:03 smithi097 ceph-mon[99491]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-08T09:53:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:03 smithi178 ceph-mon[103156]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-10-08T09:53:05.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:05 smithi186 ceph-mon[104393]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:53:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:05 smithi097 ceph-mon[99491]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:53:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:05 smithi178 ceph-mon[103156]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:53:06.425 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:06 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:53:06.425 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:06 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:06.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:06 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:53:06.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:06 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:06.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:06 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:53:06.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:06 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:07.388 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:07 smithi186 ceph-mon[104393]: Deploying daemon osd.11 on smithi186 2023-10-08T09:53:07.389 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:07 smithi186 ceph-mon[104393]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-10-08T09:53:07.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:07 smithi097 ceph-mon[99491]: Deploying daemon osd.11 on smithi186 2023-10-08T09:53:07.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:07 smithi097 ceph-mon[99491]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-10-08T09:53:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:07 smithi178 ceph-mon[103156]: Deploying daemon osd.11 on smithi186 2023-10-08T09:53:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:07 smithi178 ceph-mon[103156]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-10-08T09:53:09.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:09 smithi097 ceph-mon[99491]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:53:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:09 smithi178 ceph-mon[103156]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:53:09.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:09 smithi186 ceph-mon[104393]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-10-08T09:53:10.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:10 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:10 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:10 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:10.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:10 smithi186 ceph-mon[104393]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-08T09:53:10.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:10 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:10 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:10 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:10.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:10 smithi097 ceph-mon[99491]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-08T09:53:10.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:10 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:10 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:10.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:10 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:10.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:10 smithi178 ceph-mon[103156]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-10-08T09:53:12.196 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2023-10-08T09:53:12.196 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.195+0000 7f6059ffb700 1 -- 172.21.15.186:0/3104214757 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f605c02cfd0 con 0x7f604c0a1480 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 msgr2=0x7f604c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 0x7f604c0a3940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6048005fd0 tx=0x7f6048005dc0 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 msgr2=0x7f605c069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f605000f720 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 shutdown_connections 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f604c0a1480 0x7f604c0a3940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f605c1012f0 0x7f605c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f605c0ff750 0x7f605c069990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 --2- 172.21.15.186:0/3104214757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c100950 0x7f605c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:12.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 >> 172.21.15.186:0/3104214757 conn(0x7f605c0facf0 msgr2=0x7f605c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:12.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 shutdown_connections 2023-10-08T09:53:12.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:12.198+0000 7f6062d8c700 1 -- 172.21.15.186:0/3104214757 wait complete. 2023-10-08T09:53:12.595 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-08T09:53:12.597 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-08T09:53:12.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T09:53:12.632 INFO:journalctl@ceph.osd.11.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:53:13.166 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:53:13.343 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:53:13.343 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.343 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.361 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:53:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:13.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:13.685 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 09:53:13 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T09:53:13.683+0000 7f11a293a640 -1 osd.11 0 log_to_monitors true 2023-10-08T09:53:14.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:14 smithi097 ceph-mon[99491]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.158 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:14 smithi097 ceph-mon[99491]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 -- 172.21.15.97:0/2028114725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c0ff730 msgr2=0x7f968c0ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 --2- 172.21.15.97:0/2028114725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c0ff730 0x7f968c0ffb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f96800099c0 tx=0x7f9680009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 -- 172.21.15.97:0/2028114725 shutdown_connections 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 --2- 172.21.15.97:0/2028114725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c1012d0 0x7f968c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 --2- 172.21.15.97:0/2028114725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c0ff730 0x7f968c0ffb50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 --2- 172.21.15.97:0/2028114725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 0x7f968c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 -- 172.21.15.97:0/2028114725 >> 172.21.15.97:0/2028114725 conn(0x7f968c0facf0 msgr2=0x7f968c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 -- 172.21.15.97:0/2028114725 shutdown_connections 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.342+0000 7f96919ee700 1 -- 172.21.15.97:0/2028114725 wait complete. 2023-10-08T09:53:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 Processor -- start 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 -- start start 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c0ff730 0x7f968c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 0x7f968c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1a0540 con 0x7f968c1012d0 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1a0680 con 0x7f968c100930 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f96919ee700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1a0810 con 0x7f968c0ff730 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54720/0 (socket says 172.21.15.97:54720) 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 -- 172.21.15.97:0/1732287916 learned_addr learned my addr 172.21.15.97:0/1732287916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968affd700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c0ff730 0x7f968c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968a7fc700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 0x7f968c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 -- 172.21.15.97:0/1732287916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c0ff730 msgr2=0x7f968c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c0ff730 0x7f968c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 -- 172.21.15.97:0/1732287916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 msgr2=0x7f968c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 0x7f968c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.343+0000 7f968b7fe700 1 -- 172.21.15.97:0/1732287916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800095f0 con 0x7f968c1012d0 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f968b7fe700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9684007f10 tx=0x7f968400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684017070 con 0x7f968c1012d0 2023-10-08T09:53:14.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c1a0af0 con 0x7f968c1012d0 2023-10-08T09:53:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968400f040 con 0x7f968c1012d0 2023-10-08T09:53:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684005740 con 0x7f968c1012d0 2023-10-08T09:53:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.344+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1a1040 con 0x7f968c1012d0 2023-10-08T09:53:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.345+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c04e4c0 con 0x7f968c1012d0 2023-10-08T09:53:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.346+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f96840058a0 con 0x7f968c1012d0 2023-10-08T09:53:14.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.346+0000 7f96909ec700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 0x7f96740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:14.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.346+0000 7f968affd700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 0x7f96740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:14.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.347+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f96840cc610 con 0x7f968c1012d0 2023-10-08T09:53:14.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.347+0000 7f968affd700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 0x7f96740a39c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9680003820 tx=0x7f9680005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:14.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.350+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96840961a0 con 0x7f968c1012d0 2023-10-08T09:53:14.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:14 smithi178 ceph-mon[103156]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:14 smithi178 ceph-mon[103156]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:14 smithi186 ceph-mon[104393]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:14 smithi186 ceph-mon[104393]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:53:14.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.541+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f968c10b1b0 con 0x7f968c1012d0 2023-10-08T09:53:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.542+0000 7f96909ec700 1 -- 172.21.15.97:0/1732287916 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f96840997c0 con 0x7f968c1012d0 2023-10-08T09:53:14.543 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 msgr2=0x7f96740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 0x7f96740a39c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9680003820 tx=0x7f9680005ad0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 msgr2=0x7f968c19ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9684007f10 tx=0x7f968400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 shutdown_connections 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96740a1500 0x7f96740a39c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f968c1012d0 0x7f968c19ff70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f968c0ff730 0x7f968c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 --2- 172.21.15.97:0/1732287916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f968c100930 0x7f968c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 >> 172.21.15.97:0/1732287916 conn(0x7f968c0facf0 msgr2=0x7f968c0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 shutdown_connections 2023-10-08T09:53:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:14.544+0000 7f96919ee700 1 -- 172.21.15.97:0/1732287916 wait complete. 2023-10-08T09:53:14.923 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1696758775,"num_in_osds":12,"osd_in_since":1696758779,"num_remapped_pgs":0} 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1732287916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:15.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:15 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.275 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T09:53:15.085+0000 7f1197fb1700 -1 osd.11 0 waiting for initial osdmap 2023-10-08T09:53:15.275 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T09:53:15.098+0000 7f1192da4700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:53:15.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:15.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:53:15.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:53:15.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:15.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.276 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1732287916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.277 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:15 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1732287916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:15.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:53:15.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:15.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:15.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:15 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:15.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: Detected new or changed devices on smithi186 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:16.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: Detected new or changed devices on smithi186 2023-10-08T09:53:16.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-08T09:53:16.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:53:16.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:53:16.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:16.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:16.484 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:53:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: Detected new or changed devices on smithi186 2023-10-08T09:53:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-08T09:53:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-08T09:53:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:53:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: purged_snaps scrub starts 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: purged_snaps scrub ok 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] boot 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:53:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:17 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: purged_snaps scrub starts 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: purged_snaps scrub ok 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] boot 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:53:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:17 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: purged_snaps scrub starts 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: purged_snaps scrub ok 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: osd.11 [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] boot 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:53:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:17 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/2577391042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc41012d0 msgr2=0x7f4bc4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/2577391042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc41012d0 0x7f4bc4107b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b0d0 tx=0x7f4bbc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/2577391042 shutdown_connections 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/2577391042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc41012d0 0x7f4bc4107b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/2577391042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 0x7f4bc40ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/2577391042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc4100930 0x7f4bc4100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/2577391042 >> 172.21.15.97:0/2577391042 conn(0x7f4bc40facf0 msgr2=0x7f4bc40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/2577391042 shutdown_connections 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.662+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/2577391042 wait complete. 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 Processor -- start 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 -- start start 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 0x7f4bc4103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc41012d0 0x7f4bc4102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4102ac0 con 0x7f4bc4100930 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4102c30 con 0x7f4bc41012d0 2023-10-08T09:53:17.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bcb4d6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41a0840 con 0x7f4bc40ff730 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54736/0 (socket says 172.21.15.97:54736) 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 -- 172.21.15.97:0/385195186 learned_addr learned my addr 172.21.15.97:0/385195186 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc9272700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 0x7f4bc4103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc9a73700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc41012d0 0x7f4bc4102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:17.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 -- 172.21.15.97:0/385195186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 msgr2=0x7f4bc4103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 0x7f4bc4103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 -- 172.21.15.97:0/385195186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc41012d0 msgr2=0x7f4bc4102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc41012d0 0x7f4bc4102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.663+0000 7f4bc8a71700 1 -- 172.21.15.97:0/385195186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009d00 con 0x7f4bc4100930 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bc8a71700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c9b0 tx=0x7f4bc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007b00 con 0x7f4bc4100930 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc41a0ad0 con 0x7f4bc4100930 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bc0007c60 con 0x7f4bc4100930 2023-10-08T09:53:17.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0018660 con 0x7f4bc4100930 2023-10-08T09:53:17.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.664+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc41a0fd0 con 0x7f4bc4100930 2023-10-08T09:53:17.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.665+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc404e4c0 con 0x7f4bc4100930 2023-10-08T09:53:17.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.666+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4bc000f450 con 0x7f4bc4100930 2023-10-08T09:53:17.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.666+0000 7f4bba7fc700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 0x7f4bb00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:17.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.666+0000 7f4bc9272700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 0x7f4bb00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:17.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.667+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4bc00cc810 con 0x7f4bc4100930 2023-10-08T09:53:17.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.667+0000 7f4bc9272700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 0x7f4bb00a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4100790 tx=0x7f4bb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.671+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4bc0096310 con 0x7f4bc4100930 2023-10-08T09:53:17.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.859+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f4bc410af90 con 0x7f4bc4100930 2023-10-08T09:53:17.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.859+0000 7f4bba7fc700 1 -- 172.21.15.97:0/385195186 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4bc0099930 con 0x7f4bc4100930 2023-10-08T09:53:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 msgr2=0x7f4bb00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 0x7f4bb00a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4100790 tx=0x7f4bb400b410 comp rx=0 tx=0).stop 2023-10-08T09:53:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 msgr2=0x7f4bc4101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c9b0 tx=0x7f4bc000ccc0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 shutdown_connections 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bb00a1480 0x7f4bb00a3940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bc41012d0 0x7f4bc4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bc40ff730 0x7f4bc4103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 --2- 172.21.15.97:0/385195186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bc4100930 0x7f4bc4101de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.861+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 >> 172.21.15.97:0/385195186 conn(0x7f4bc40facf0 msgr2=0x7f4bc4068190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.862+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 shutdown_connections 2023-10-08T09:53:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:17.862+0000 7f4bcb4d6700 1 -- 172.21.15.97:0/385195186 wait complete. 2023-10-08T09:53:18.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:18 smithi097 ceph-mon[99491]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:53:18.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/385195186' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:18.230 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1696758796,"num_in_osds":12,"osd_in_since":1696758779,"num_remapped_pgs":0} 2023-10-08T09:53:18.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:53:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:18 smithi178 ceph-mon[103156]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:53:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/385195186' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:18 smithi186 ceph-mon[104393]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:53:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/385195186' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:53:18.787 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:53:19.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:19 smithi097 ceph-mon[99491]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:19 smithi178 ceph-mon[103156]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:19 smithi186 ceph-mon[104393]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 -- 172.21.15.97:0/723983601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd4101260 msgr2=0x7f4bd410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/723983601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd4101260 0x7f4bd410d650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b0a0 tx=0x7f4bcc00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 -- 172.21.15.97:0/723983601 shutdown_connections 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/723983601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd4101260 0x7f4bd410d650 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/723983601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd41057c0 0x7f4bd4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/723983601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 -- 172.21.15.97:0/723983601 >> 172.21.15.97:0/723983601 conn(0x7f4bd40779e0 msgr2=0x7f4bd4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 -- 172.21.15.97:0/723983601 shutdown_connections 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.961+0000 7f4bd98e2700 1 -- 172.21.15.97:0/723983601 wait complete. 2023-10-08T09:53:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 Processor -- start 2023-10-08T09:53:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 -- start start 2023-10-08T09:53:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 0x7f4bd4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419bcb0 con 0x7f4bd41057c0 2023-10-08T09:53:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419be20 con 0x7f4bd4101260 2023-10-08T09:53:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.962+0000 7f4bd98e2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd419bfe0 con 0x7f4bd41008a0 2023-10-08T09:53:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54756/0 (socket says 172.21.15.97:54756) 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:56452/0 (socket says 172.21.15.97:56452) 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 -- 172.21.15.97:0/2881751383 learned_addr learned my addr 172.21.15.97:0/2881751383 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd27fc700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 0x7f4bd4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 -- 172.21.15.97:0/2881751383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 msgr2=0x7f4bd4196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 -- 172.21.15.97:0/2881751383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 msgr2=0x7f4bd4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 0x7f4bd4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 -- 172.21.15.97:0/2881751383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc009d00 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd27fc700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 0x7f4bd4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd37fe700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b830 tx=0x7f4bcc007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00e040 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd419c260 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bcc004c60 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc005280 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.963+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd41a3b20 con 0x7f4bd41057c0 2023-10-08T09:53:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.964+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4108d80 con 0x7f4bd41057c0 2023-10-08T09:53:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.965+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4bcc01b450 con 0x7f4bd41057c0 2023-10-08T09:53:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.966+0000 7f4bd88e0700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 0x7f4bbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.966+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4bcc0ce6f0 con 0x7f4bd41057c0 2023-10-08T09:53:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.966+0000 7f4bd2ffd700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 0x7f4bbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.966+0000 7f4bd2ffd700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 0x7f4bbc0a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4101930 tx=0x7f4bc8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:19.969+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4bcc098280 con 0x7f4bd41057c0 2023-10-08T09:53:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.160+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd41980c0 con 0x7f4bd41057c0 2023-10-08T09:53:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.161+0000 7f4bd88e0700 1 -- 172.21.15.97:0/2881751383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18435 (secure 0 0 0) 0x7f4bcc09b8a0 con 0x7f4bd41057c0 2023-10-08T09:53:20.162 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:20.163 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":77,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:53:17.122023+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:53:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 msgr2=0x7f4bbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 0x7f4bbc0a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4101930 tx=0x7f4bc8009190 comp rx=0 tx=0).stop 2023-10-08T09:53:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 msgr2=0x7f4bd419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b830 tx=0x7f4bcc007870 comp rx=0 tx=0).stop 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 shutdown_connections 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4bbc0a1480 0x7f4bbc0a3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd41057c0 0x7f4bd419b6a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4bd41008a0 0x7f4bd4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 --2- 172.21.15.97:0/2881751383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4bd4101260 0x7f4bd4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.163+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 >> 172.21.15.97:0/2881751383 conn(0x7f4bd40779e0 msgr2=0x7f4bd40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.164+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 shutdown_connections 2023-10-08T09:53:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:20.164+0000 7f4bd98e2700 1 -- 172.21.15.97:0/2881751383 wait complete. 2023-10-08T09:53:20.531 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:53:20.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-08T09:53:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:53:21.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:21 smithi097 ceph-mon[99491]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:21.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:21 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2881751383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:21 smithi178 ceph-mon[103156]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:21 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2881751383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:21 smithi186 ceph-mon[104393]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:21 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2881751383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- 172.21.15.97:0/359052910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c1045b0 msgr2=0x7fb49c104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/359052910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c1045b0 0x7fb49c104990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb490009990 tx=0x7fb490009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- 172.21.15.97:0/359052910 shutdown_connections 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/359052910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb49c1000a0 0x7fb49c10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/359052910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c1045b0 0x7fb49c104990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/359052910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c0ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- 172.21.15.97:0/359052910 >> 172.21.15.97:0/359052910 conn(0x7fb49c0766c0 msgr2=0x7fb49c076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- 172.21.15.97:0/359052910 shutdown_connections 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- 172.21.15.97:0/359052910 wait complete. 2023-10-08T09:53:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 Processor -- start 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.303+0000 7fb4a30e1700 1 -- start start 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 0x7fb49c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb49c1045b0 0x7fb49c1a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c198920 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1a30e0 con 0x7fb49c1000a0 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1a3220 con 0x7fb49c1045b0 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54764/0 (socket says 172.21.15.97:54764) 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 -- 172.21.15.97:0/2602712258 learned_addr learned my addr 172.21.15.97:0/2602712258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb49bfff700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 0x7fb49c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 -- 172.21.15.97:0/2602712258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb49c1045b0 msgr2=0x7fb49c1a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb49c1045b0 0x7fb49c1a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 -- 172.21.15.97:0/2602712258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 msgr2=0x7fb49c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 0x7fb49c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 -- 172.21.15.97:0/2602712258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb49bfff700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 0x7fb49c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a0e7d700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb490000c00 tx=0x7fb49000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49001d070 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c1a3440 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49000fca0 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.304+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490017750 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.305+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c1a38e0 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.305+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c107c20 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.307+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb490022470 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.307+0000 7fb499ffb700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 0x7fb48c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.307+0000 7fb49bfff700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 0x7fb48c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.307+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4900ce180 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.308+0000 7fb49bfff700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 0x7fb48c0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb49c069270 tx=0x7fb488008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.311+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb490097d10 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.501+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb49c069020 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.502+0000 7fb499ffb700 1 -- 172.21.15.97:0/2602712258 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb49009b330 con 0x7fb49c0ff6e0 2023-10-08T09:53:22.503 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:53:22.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 msgr2=0x7fb48c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 0x7fb48c0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb49c069270 tx=0x7fb488008040 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 msgr2=0x7fb49c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb490000c00 tx=0x7fb49000f7b0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 shutdown_connections 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb48c0a1480 0x7fb48c0a3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb49c1045b0 0x7fb49c1a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49c0ff6e0 0x7fb49c197b20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 --2- 172.21.15.97:0/2602712258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb49c1000a0 0x7fb49c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 >> 172.21.15.97:0/2602712258 conn(0x7fb49c0766c0 msgr2=0x7fb49c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 shutdown_connections 2023-10-08T09:53:22.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:22.504+0000 7fb4a30e1700 1 -- 172.21.15.97:0/2602712258 wait complete. 2023-10-08T09:53:22.843 INFO:tasks.cephadm:Adding mds.a on smithi097 2023-10-08T09:53:22.843 INFO:tasks.cephadm:Adding mds.d on smithi097 2023-10-08T09:53:22.843 INFO:tasks.cephadm:Adding mds.g on smithi097 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.j on smithi097 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.b on smithi178 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.e on smithi178 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.h on smithi178 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.k on smithi178 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.c on smithi186 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.f on smithi186 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.i on smithi186 2023-10-08T09:53:22.844 INFO:tasks.cephadm:Adding mds.l on smithi186 2023-10-08T09:53:22.844 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2023-10-08T09:53:23.418 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:53:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:23 smithi097 ceph-mon[99491]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:23 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2602712258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:23 smithi178 ceph-mon[103156]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:23 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2602712258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:23 smithi186 ceph-mon[104393]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:23 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2602712258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.520+0000 7f45b7155700 1 -- 172.21.15.186:0/2397087173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 msgr2=0x7f45b00ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.520+0000 7f45b7155700 1 --2- 172.21.15.186:0/2397087173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b00ffb30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009990 tx=0x7f45a4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- 172.21.15.186:0/2397087173 shutdown_connections 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- 172.21.15.186:0/2397087173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b01012b0 0x7f45b0107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- 172.21.15.186:0/2397087173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b00ffb30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- 172.21.15.186:0/2397087173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b0100910 0x7f45b0100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.523 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- 172.21.15.186:0/2397087173 >> 172.21.15.186:0/2397087173 conn(0x7f45b00facb0 msgr2=0x7f45b00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- 172.21.15.186:0/2397087173 shutdown_connections 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- 172.21.15.186:0/2397087173 wait complete. 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 Processor -- start 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- start start 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 0x7f45b0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b01012b0 0x7f45b019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a04e0 con 0x7f45b0100910 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a0620 con 0x7f45b01012b0 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b7155700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01a07b0 con 0x7f45b00ff710 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b4ef1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:24.524 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b4ef1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57374/0 (socket says 172.21.15.186:57374) 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.521+0000 7f45b4ef1700 1 -- 172.21.15.186:0/800126234 learned_addr learned my addr 172.21.15.186:0/800126234 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45affff700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 0x7f45b0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b56f2700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b01012b0 0x7f45b019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 -- 172.21.15.186:0/800126234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b01012b0 msgr2=0x7f45b019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b01012b0 0x7f45b019ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 -- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 msgr2=0x7f45b0198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 0x7f45b0198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 -- 172.21.15.186:0/800126234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40095f0 con 0x7f45b00ff710 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45affff700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 0x7f45b0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b4ef1700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45a400f670 tx=0x7f45a400f750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a401d070 con 0x7f45b00ff710 2023-10-08T09:53:24.525 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a400fcf0 con 0x7f45b00ff710 2023-10-08T09:53:24.526 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b01a0a30 con 0x7f45b00ff710 2023-10-08T09:53:24.526 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4017700 con 0x7f45b00ff710 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.522+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b01a0ea0 con 0x7f45b00ff710 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.523+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b004e4c0 con 0x7f45b00ff710 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.524+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f45a4017860 con 0x7f45b00ff710 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.525+0000 7f45adffb700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 0x7f45a00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.525+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45a40cdbc0 con 0x7f45b00ff710 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.525+0000 7f45affff700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 0x7f45a00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:24.527 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.526+0000 7f45affff700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 0x7f45a00a39c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f459c0099d0 tx=0x7f459c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:24.530 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.529+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f45a405d100 con 0x7f45b00ff710 2023-10-08T09:53:24.729 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.729+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f45b00fcfd0 con 0x7f45a00a1500 2023-10-08T09:53:24.737 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.737+0000 7f45adffb700 1 -- 172.21.15.186:0/800126234 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f45b00fcfd0 con 0x7f45a00a1500 2023-10-08T09:53:24.737 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 msgr2=0x7f45a00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 0x7f45a00a39c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f459c0099d0 tx=0x7f459c008040 comp rx=0 tx=0).stop 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 msgr2=0x7f45b01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45a400f670 tx=0x7f45a400f750 comp rx=0 tx=0).stop 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 shutdown_connections 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f45a00a1500 0x7f45a00a39c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45b01012b0 0x7f45b019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45b00ff710 0x7f45b01989a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.740 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 --2- 172.21.15.186:0/800126234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b0100910 0x7f45b0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:24.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 >> 172.21.15.186:0/800126234 conn(0x7f45b00facb0 msgr2=0x7f45b00fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:24.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 shutdown_connections 2023-10-08T09:53:24.741 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-08T09:53:24.739+0000 7f45b7155700 1 -- 172.21.15.186:0/800126234 wait complete. 2023-10-08T09:53:25.078 DEBUG:teuthology.orchestra.run.smithi097:mds.a> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-08T09:53:25.080 DEBUG:teuthology.orchestra.run.smithi097:mds.d> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-08T09:53:25.117 INFO:journalctl@ceph.mds.a.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:53:25.118 DEBUG:teuthology.orchestra.run.smithi097:mds.g> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-08T09:53:25.148 INFO:journalctl@ceph.mds.d.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:53:25.149 DEBUG:teuthology.orchestra.run.smithi097:mds.j> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-08T09:53:25.177 INFO:journalctl@ceph.mds.g.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:53:25.178 DEBUG:teuthology.orchestra.run.smithi178:mds.b> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-08T09:53:25.180 DEBUG:teuthology.orchestra.run.smithi178:mds.e> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-08T09:53:25.204 INFO:journalctl@ceph.mds.j.smithi097.stdout:-- Logs begin at Sun 2023-10-08 09:32:28 UTC. -- 2023-10-08T09:53:25.217 INFO:journalctl@ceph.mds.b.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:53:25.218 DEBUG:teuthology.orchestra.run.smithi178:mds.h> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-08T09:53:25.247 INFO:journalctl@ceph.mds.e.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:53:25.248 DEBUG:teuthology.orchestra.run.smithi178:mds.k> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-08T09:53:25.276 INFO:journalctl@ceph.mds.h.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:53:25.277 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-08T09:53:25.279 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-08T09:53:25.303 INFO:journalctl@ceph.mds.k.smithi178.stdout:-- Logs begin at Sun 2023-10-08 09:33:03 UTC. -- 2023-10-08T09:53:25.308 INFO:journalctl@ceph.mds.c.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:53:25.309 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:25.334 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:25.335 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:25 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.335 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-08T09:53:25.336 INFO:journalctl@ceph.mds.f.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:53:25.357 INFO:journalctl@ceph.mds.i.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:53:25.358 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-08T09:53:25.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:25.384 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:25 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.386 INFO:journalctl@ceph.mds.l.smithi186.stdout:-- Logs begin at Sun 2023-10-08 09:32:49 UTC. -- 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:25.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:25.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:25.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:25 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.490+0000 7fe6755e0700 1 -- 172.21.15.97:0/2821425997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670101130 msgr2=0x7fe6701079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.490+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2821425997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670101130 0x7fe6701079e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe66800b0d0 tx=0x7fe66800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.490+0000 7fe6755e0700 1 -- 172.21.15.97:0/2821425997 shutdown_connections 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2821425997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670101130 0x7fe6701079e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2821425997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 0x7fe6700ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2821425997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670100790 0x7fe670100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 -- 172.21.15.97:0/2821425997 >> 172.21.15.97:0/2821425997 conn(0x7fe6700fab10 msgr2=0x7fe6700fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 -- 172.21.15.97:0/2821425997 shutdown_connections 2023-10-08T09:53:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 -- 172.21.15.97:0/2821425997 wait complete. 2023-10-08T09:53:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.491+0000 7fe6755e0700 1 Processor -- start 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 -- start start 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 0x7fe670068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670101130 0x7fe67019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67019e1f0 con 0x7fe670100790 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67019e360 con 0x7fe670101130 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe6755e0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67019e520 con 0x7fe6700ff5a0 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54778/0 (socket says 172.21.15.97:54778) 2023-10-08T09:53:25.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 -- 172.21.15.97:0/2102377728 learned_addr learned my addr 172.21.15.97:0/2102377728 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66f7fe700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670101130 0x7fe67019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66effd700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 0x7fe670068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 -- 172.21.15.97:0/2102377728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 msgr2=0x7fe670068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 0x7fe670068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 -- 172.21.15.97:0/2102377728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670101130 msgr2=0x7fe67019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670101130 0x7fe67019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 -- 172.21.15.97:0/2102377728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668009d00 con 0x7fe670100790 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe66e7fc700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9a0 tx=0x7fe65800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.492+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658009790 con 0x7fe670100790 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.493+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67019e800 con 0x7fe670100790 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.493+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe658010430 con 0x7fe670100790 2023-10-08T09:53:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.493+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658018680 con 0x7fe670100790 2023-10-08T09:53:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.493+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67019ed10 con 0x7fe670100790 2023-10-08T09:53:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.493+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe670106950 con 0x7fe670100790 2023-10-08T09:53:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.494+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe6580105a0 con 0x7fe670100790 2023-10-08T09:53:25.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.495+0000 7fe667fff700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 0x7fe65c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.495+0000 7fe66effd700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 0x7fe65c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.495+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe658014070 con 0x7fe670100790 2023-10-08T09:53:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.495+0000 7fe66effd700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 0x7fe65c0a3940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe660005950 tx=0x7fe66000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.499+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe658096e00 con 0x7fe670100790 2023-10-08T09:53:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.693+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe670067450 con 0x7fe670100790 2023-10-08T09:53:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.694+0000 7fe667fff700 1 -- 172.21.15.97:0/2102377728 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18435 (secure 0 0 0) 0x7fe65809a420 con 0x7fe670100790 2023-10-08T09:53:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:25.695 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":77,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:53:17.122023+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 msgr2=0x7fe65c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 0x7fe65c0a3940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe660005950 tx=0x7fe66000a300 comp rx=0 tx=0).stop 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 msgr2=0x7fe67019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9a0 tx=0x7fe65800ecb0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 shutdown_connections 2023-10-08T09:53:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe65c0a1480 0x7fe65c0a3940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe670101130 0x7fe67019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6700ff5a0 0x7fe670068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 --2- 172.21.15.97:0/2102377728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe670100790 0x7fe67019b460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 >> 172.21.15.97:0/2102377728 conn(0x7fe6700fab10 msgr2=0x7fe670103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 shutdown_connections 2023-10-08T09:53:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.696+0000 7fe6755e0700 1 -- 172.21.15.97:0/2102377728 wait complete. 2023-10-08T09:53:25.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:53:25.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:53:25.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 -- 172.21.15.97:0/219612122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2581002b0 msgr2=0x7fc258107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/219612122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2581002b0 0x7fc258107b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc25000b0a0 tx=0x7fc25000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 -- 172.21.15.97:0/219612122 shutdown_connections 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/219612122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2581002b0 0x7fc258107b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/219612122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 0x7fc2581010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/219612122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 -- 172.21.15.97:0/219612122 >> 172.21.15.97:0/219612122 conn(0x7fc2580fa3c0 msgr2=0x7fc2580fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 -- 172.21.15.97:0/219612122 shutdown_connections 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.873+0000 7fc25d3cc700 1 -- 172.21.15.97:0/219612122 wait complete. 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 Processor -- start 2023-10-08T09:53:25.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 -- start start 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2581002b0 0x7fc25819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 0x7fc25819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25819e260 con 0x7fc2580ff8f0 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25819e3d0 con 0x7fc258100ca0 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc25d3cc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25819e590 con 0x7fc2581002b0 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc256ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc256ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54788/0 (socket says 172.21.15.97:54788) 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc256ffd700 1 -- 172.21.15.97:0/1386612419 learned_addr learned my addr 172.21.15.97:0/1386612419 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:25.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.874+0000 7fc2567fc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2581002b0 0x7fc25819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc2577fe700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 0x7fc25819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 -- 172.21.15.97:0/1386612419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2581002b0 msgr2=0x7fc25819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2581002b0 0x7fc25819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 -- 172.21.15.97:0/1386612419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 msgr2=0x7fc25819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 0x7fc25819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 -- 172.21.15.97:0/1386612419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc250009d00 con 0x7fc2580ff8f0 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc256ffd700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c990 tx=0x7fc24800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248007b00 con 0x7fc2580ff8f0 2023-10-08T09:53:25.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc248007c60 con 0x7fc2580ff8f0 2023-10-08T09:53:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25819e870 con 0x7fc2580ff8f0 2023-10-08T09:53:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248018660 con 0x7fc2580ff8f0 2023-10-08T09:53:25.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.875+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25819edd0 con 0x7fc2580ff8f0 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.876+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc258106ac0 con 0x7fc2580ff8f0 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.877+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc24800f450 con 0x7fc2580ff8f0 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.878+0000 7fc23ffff700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 0x7fc2400a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.878+0000 7fc2567fc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 0x7fc2400a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.878+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2480cc860 con 0x7fc2580ff8f0 2023-10-08T09:53:25.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.878+0000 7fc2567fc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 0x7fc2400a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ff8c0 tx=0x7fc24c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:25.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:25.881+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc2480963e0 con 0x7fc2580ff8f0 2023-10-08T09:53:26.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.072+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc25819ea00 con 0x7fc2580ff8f0 2023-10-08T09:53:26.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.072+0000 7fc23ffff700 1 -- 172.21.15.97:0/1386612419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc25819ea00 con 0x7fc2580ff8f0 2023-10-08T09:53:26.073 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 msgr2=0x7fc2400a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 0x7fc2400a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc2580ff8c0 tx=0x7fc24c007400 comp rx=0 tx=0).stop 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 msgr2=0x7fc2580ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c990 tx=0x7fc24800cca0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 shutdown_connections 2023-10-08T09:53:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2400a1500 0x7fc2400a39c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc258100ca0 0x7fc25819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2580ff8f0 0x7fc2580ff360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 --2- 172.21.15.97:0/1386612419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2581002b0 0x7fc25819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 >> 172.21.15.97:0/1386612419 conn(0x7fc2580fa3c0 msgr2=0x7fc258103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 shutdown_connections 2023-10-08T09:53:26.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.074+0000 7fc25d3cc700 1 -- 172.21.15.97:0/1386612419 wait complete. 2023-10-08T09:53:26.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2883843503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a34101a20 msgr2=0x7f7a34101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2883843503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a34101a20 0x7f7a34101e80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009990 tx=0x7f7a30009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2883843503 shutdown_connections 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2883843503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a341023c0 0x7f7a34108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2883843503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a340fe9b0 0x7f7a340fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2883843503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a34101a20 0x7f7a34101e80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2883843503 >> 172.21.15.97:0/2883843503 conn(0x7f7a340fa2c0 msgr2=0x7f7a340fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2883843503 shutdown_connections 2023-10-08T09:53:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2883843503 wait complete. 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 Processor -- start 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.209+0000 7f7a3c3fd700 1 -- start start 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a34101a20 0x7f7a34196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a341023c0 0x7f7a3419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3419e150 con 0x7f7a340fe9b0 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3419e290 con 0x7f7a34101a20 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3419e420 con 0x7f7a341023c0 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54802/0 (socket says 172.21.15.97:54802) 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 -- 172.21.15.97:0/2575961817 learned_addr learned my addr 172.21.15.97:0/2575961817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 -- 172.21.15.97:0/2575961817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a341023c0 msgr2=0x7f7a3419db80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a341023c0 0x7f7a3419db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 -- 172.21.15.97:0/2575961817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a34101a20 msgr2=0x7f7a34196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a34101a20 0x7f7a34196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 -- 172.21.15.97:0/2575961817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300095f0 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3a199700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400d980 tx=0x7f7a2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400bb60 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3419e700 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a24007620 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400f620 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.210+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3419ec10 con 0x7f7a340fe9b0 2023-10-08T09:53:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.211+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34107be0 con 0x7f7a340fe9b0 2023-10-08T09:53:26.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.213+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7a24010460 con 0x7f7a340fe9b0 2023-10-08T09:53:26.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.213+0000 7f7a2b7fe700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 0x7f7a200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:26.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.213+0000 7f7a39998700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 0x7f7a200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:26.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.213+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a240cd120 con 0x7f7a340fe9b0 2023-10-08T09:53:26.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.214+0000 7f7a39998700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 0x7f7a200a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a300098e0 tx=0x7f7a30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.217+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7a24096c20 con 0x7f7a340fe9b0 2023-10-08T09:53:26.272 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 systemd[1]: Starting Ceph mds.l for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:26.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mon[104393]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:53:26.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mon[104393]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-08T09:53:26.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mon[104393]: Deploying daemon mds.l on smithi186 2023-10-08T09:53:26.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2102377728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:26.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1386612419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:26.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.451+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a340678a0 con 0x7f7a340fe9b0 2023-10-08T09:53:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.451+0000 7f7a2b7fe700 1 -- 172.21.15.97:0/2575961817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f7a2400de60 con 0x7f7a340fe9b0 2023-10-08T09:53:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:26.453 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-08T09:53:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:26 smithi097 ceph-mon[99491]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:53:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:26 smithi097 ceph-mon[99491]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-08T09:53:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:26 smithi097 ceph-mon[99491]: Deploying daemon mds.l on smithi186 2023-10-08T09:53:26.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:26 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2102377728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:26.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:26 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1386612419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:26.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 msgr2=0x7f7a200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 0x7f7a200a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a300098e0 tx=0x7f7a30005fb0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 msgr2=0x7f7a34196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400d980 tx=0x7f7a2400dd40 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 shutdown_connections 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a200a1480 0x7f7a200a3940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a341023c0 0x7f7a3419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a340fe9b0 0x7f7a34196620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 --2- 172.21.15.97:0/2575961817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a34101a20 0x7f7a34196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.453+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 >> 172.21.15.97:0/2575961817 conn(0x7f7a340fa2c0 msgr2=0x7f7a34104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:26.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.454+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 shutdown_connections 2023-10-08T09:53:26.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:26.454+0000 7f7a3c3fd700 1 -- 172.21.15.97:0/2575961817 wait complete. 2023-10-08T09:53:26.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:26 smithi178 ceph-mon[103156]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:53:26.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:26 smithi178 ceph-mon[103156]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-08T09:53:26.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:26 smithi178 ceph-mon[103156]: Deploying daemon mds.l on smithi186 2023-10-08T09:53:26.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:26 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2102377728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:26.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:26 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1386612419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:26.457 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 1 2023-10-08T09:53:26.849 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 podman[141745]: 2023-10-08T09:53:27.147 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 bash[141745]: d563dcb7c40a45f0a3d9e129118208c7bdbc03f851882aad42ecf8e04be53ce6 2023-10-08T09:53:27.148 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-mds[141785]: -- 172.21.15.186:0/635924474 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591737d91e0 con 0x5591737ae800 2023-10-08T09:53:27.148 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-l[141761]: starting mds.l at 2023-10-08T09:53:27.148 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 09:53:26 smithi186 systemd[1]: Started Ceph mds.l for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:27.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:27.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2575961817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:27.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6832/3404847957,v1:172.21.15.186:6833/3404847957] up:boot 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: fsmap 1 up:standby 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:27.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:27 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2575961817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6832/3404847957,v1:172.21.15.186:6833/3404847957] up:boot 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: fsmap 1 up:standby 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:27.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:27.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:27 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2575961817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6832/3404847957,v1:172.21.15.186:6833/3404847957] up:boot 2023-10-08T09:53:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: fsmap 1 up:standby 2023-10-08T09:53:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:27.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:27 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:28.172 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:28 smithi178 ceph-mon[103156]: Deploying daemon mds.k on smithi178 2023-10-08T09:53:28.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:28 smithi097 ceph-mon[99491]: Deploying daemon mds.k on smithi178 2023-10-08T09:53:28.455 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:28 smithi178 systemd[1]: Starting Ceph mds.k for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:28.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:28 smithi186 ceph-mon[104393]: Deploying daemon mds.k on smithi178 2023-10-08T09:53:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 -- 172.21.15.97:0/1397980661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 msgr2=0x7f58c0105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 --2- 172.21.15.97:0/1397980661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0105ea0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009990 tx=0x7f58bc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 -- 172.21.15.97:0/1397980661 shutdown_connections 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 --2- 172.21.15.97:0/1397980661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c01063e0 0x7f58c01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 --2- 172.21.15.97:0/1397980661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0101180 0x7f58c0103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 --2- 172.21.15.97:0/1397980661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0105ea0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 -- 172.21.15.97:0/1397980661 >> 172.21.15.97:0/1397980661 conn(0x7f58c00fab10 msgr2=0x7f58c00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 -- 172.21.15.97:0/1397980661 shutdown_connections 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.602+0000 7f58c820a700 1 -- 172.21.15.97:0/1397980661 wait complete. 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 Processor -- start 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 -- start start 2023-10-08T09:53:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c0101180 0x7f58c0196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c01063e0 0x7f58c019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e070 con 0x7f58c0103ab0 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e1b0 con 0x7f58c01063e0 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c820a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c019e340 con 0x7f58c0101180 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c57a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c57a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54818/0 (socket says 172.21.15.97:54818) 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c57a5700 1 -- 172.21.15.97:0/4293472470 learned_addr learned my addr 172.21.15.97:0/4293472470 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c67a7700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c01063e0 0x7f58c019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.603+0000 7f58c5fa6700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c0101180 0x7f58c0196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 -- 172.21.15.97:0/4293472470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c0101180 msgr2=0x7f58c0196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c0101180 0x7f58c0196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 -- 172.21.15.97:0/4293472470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c01063e0 msgr2=0x7f58c019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c01063e0 0x7f58c019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 -- 172.21.15.97:0/4293472470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0095f0 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c57a5700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0095c0 tx=0x7f58bc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc01d070 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58bc005510 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c019e5c0 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004c20 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.604+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c019ea40 con 0x7f58c0103ab0 2023-10-08T09:53:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.605+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c00fc690 con 0x7f58c0103ab0 2023-10-08T09:53:28.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.606+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f58bc00fb90 con 0x7f58c0103ab0 2023-10-08T09:53:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.606+0000 7f58b6ffd700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 0x7f58ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.606+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58bc0cdb30 con 0x7f58c0103ab0 2023-10-08T09:53:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.606+0000 7f58c5fa6700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 0x7f58ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.607+0000 7f58c5fa6700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 0x7f58ac0a39c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f58b000bbd0 tx=0x7f58b000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.610+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58bc0976c0 con 0x7f58c0103ab0 2023-10-08T09:53:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.841+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c0067450 con 0x7f58c0103ab0 2023-10-08T09:53:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.842+0000 7f58b6ffd700 1 -- 172.21.15.97:0/4293472470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f58bc09ace0 con 0x7f58c0103ab0 2023-10-08T09:53:28.843 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:28.843 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:53:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 msgr2=0x7f58ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 0x7f58ac0a39c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f58b000bbd0 tx=0x7f58b000b5b0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 msgr2=0x7f58c0196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0095c0 tx=0x7f58bc00f690 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 shutdown_connections 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f58ac0a1500 0x7f58ac0a39c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c01063e0 0x7f58c019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58c0101180 0x7f58c0196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 --2- 172.21.15.97:0/4293472470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58c0103ab0 0x7f58c0196a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 >> 172.21.15.97:0/4293472470 conn(0x7f58c00fab10 msgr2=0x7f58c0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 shutdown_connections 2023-10-08T09:53:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:28.844+0000 7f58c820a700 1 -- 172.21.15.97:0/4293472470 wait complete. 2023-10-08T09:53:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2023-10-08T09:53:29.000 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:28 smithi178 podman[139303]: 2023-10-08T09:53:29.001 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:28 smithi178 ceph-mds[139343]: -- 172.21.15.178:0/1332866301 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608bff511e0 con 0x5608bff27000 2023-10-08T09:53:29.001 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:28 smithi178 bash[139303]: fbdf38855a4f00d67dcc9d0d833e251093922589e562e74f46f2291155e48463 2023-10-08T09:53:29.001 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:28 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-k[139319]: starting mds.k at 2023-10-08T09:53:29.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4293472470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:29.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 09:53:29 smithi178 systemd[1]: Started Ceph mds.k for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4293472470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4293472470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:29.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:29.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:30.399 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:30 smithi097 ceph-mon[99491]: Deploying daemon mds.j on smithi097 2023-10-08T09:53:30.399 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:30 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6832/1070195466,v1:172.21.15.178:6833/1070195466] up:boot 2023-10-08T09:53:30.399 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:30 smithi097 ceph-mon[99491]: fsmap 2 up:standby 2023-10-08T09:53:30.399 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:53:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:30 smithi178 ceph-mon[103156]: Deploying daemon mds.j on smithi097 2023-10-08T09:53:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:30 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6832/1070195466,v1:172.21.15.178:6833/1070195466] up:boot 2023-10-08T09:53:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:30 smithi178 ceph-mon[103156]: fsmap 2 up:standby 2023-10-08T09:53:30.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:53:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:30 smithi186 ceph-mon[104393]: Deploying daemon mds.j on smithi097 2023-10-08T09:53:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:30 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6832/1070195466,v1:172.21.15.178:6833/1070195466] up:boot 2023-10-08T09:53:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:30 smithi186 ceph-mon[104393]: fsmap 2 up:standby 2023-10-08T09:53:30.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:53:30.703 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:30 smithi097 systemd[1]: Starting Ceph mds.j for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:30.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- 172.21.15.97:0/1999531053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb48106320 msgr2=0x7fcb48108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 --2- 172.21.15.97:0/1999531053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb48106320 0x7fcb48108790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000b0a0 tx=0x7fcb4000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- 172.21.15.97:0/1999531053 shutdown_connections 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 --2- 172.21.15.97:0/1999531053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb48106320 0x7fcb48108790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 --2- 172.21.15.97:0/1999531053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb481010c0 0x7fcb481034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 --2- 172.21.15.97:0/1999531053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 0x7fcb48105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- 172.21.15.97:0/1999531053 >> 172.21.15.97:0/1999531053 conn(0x7fcb480faa90 msgr2=0x7fcb480fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- 172.21.15.97:0/1999531053 shutdown_connections 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- 172.21.15.97:0/1999531053 wait complete. 2023-10-08T09:53:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 Processor -- start 2023-10-08T09:53:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 -- start start 2023-10-08T09:53:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.990+0000 7fcb4d743700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 0x7fcb48196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48106320 0x7fcb4819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4819e320 con 0x7fcb481010c0 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4819e460 con 0x7fcb481039f0 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4819e5f0 con 0x7fcb48106320 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54830/0 (socket says 172.21.15.97:54830) 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 -- 172.21.15.97:0/2211539689 learned_addr learned my addr 172.21.15.97:0/2211539689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb477fe700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48106320 0x7fcb4819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb467fc700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 0x7fcb48196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 -- 172.21.15.97:0/2211539689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48106320 msgr2=0x7fcb4819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:30.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48106320 0x7fcb4819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 -- 172.21.15.97:0/2211539689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 msgr2=0x7fcb48196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 0x7fcb48196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 -- 172.21.15.97:0/2211539689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009d00 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb46ffd700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b530 tx=0x7fcb3800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011830 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4819e8d0 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb38011e50 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800f2e0 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.991+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4819eeb0 con 0x7fcb481010c0 2023-10-08T09:53:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.992+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb480fc610 con 0x7fcb481010c0 2023-10-08T09:53:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.993+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcb38010580 con 0x7fcb481010c0 2023-10-08T09:53:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.994+0000 7fcb2ffff700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 0x7fcb300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.994+0000 7fcb467fc700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 0x7fcb300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.994+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb380cc8a0 con 0x7fcb481010c0 2023-10-08T09:53:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.995+0000 7fcb467fc700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 0x7fcb300a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006fd0 tx=0x7fcb3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:30.999+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb380963a0 con 0x7fcb481010c0 2023-10-08T09:53:31.166 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:30 smithi097 podman[154568]: 2023-10-08T09:53:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.257+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb480673e0 con 0x7fcb481010c0 2023-10-08T09:53:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.260+0000 7fcb2ffff700 1 -- 172.21.15.97:0/2211539689 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fcb380999c0 con 0x7fcb481010c0 2023-10-08T09:53:31.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:31.261 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:53:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 msgr2=0x7fcb300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 0x7fcb300a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006fd0 tx=0x7fcb3c008040 comp rx=0 tx=0).stop 2023-10-08T09:53:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 msgr2=0x7fcb481967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b530 tx=0x7fcb3800b8f0 comp rx=0 tx=0).stop 2023-10-08T09:53:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 shutdown_connections 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb300a1480 0x7fcb300a3940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb48106320 0x7fcb4819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb481010c0 0x7fcb481967f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 --2- 172.21.15.97:0/2211539689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb481039f0 0x7fcb48196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 >> 172.21.15.97:0/2211539689 conn(0x7fcb480faa90 msgr2=0x7fcb48101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 shutdown_connections 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:31.262+0000 7fcb4d743700 1 -- 172.21.15.97:0/2211539689 wait complete. 2023-10-08T09:53:31.264 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2023-10-08T09:53:31.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:31 smithi097 ceph-mon[99491]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:31.453 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:31 smithi097 ceph-mds[154652]: -- 172.21.15.97:0/2771152795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ec0ccbb1e0 con 0x55ec0cc91000 2023-10-08T09:53:31.453 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:31 smithi097 bash[154568]: cfa9ad3e684359b2de3ea42dee75e9f4eb23cc06a2f26206d5db2cf4f3154416 2023-10-08T09:53:31.453 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:31 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-j[154608]: starting mds.j at 2023-10-08T09:53:31.453 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 09:53:31 smithi097 systemd[1]: Started Ceph mds.j for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:31 smithi178 ceph-mon[103156]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:31.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:31 smithi186 ceph-mon[104393]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2211539689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: Deploying daemon mds.d on smithi097 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.97:6834/2090847706,v1:172.21.15.97:6835/2090847706] up:boot 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: fsmap 3 up:standby 2023-10-08T09:53:32.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:32 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2211539689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: Deploying daemon mds.d on smithi097 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:32.572 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.97:6834/2090847706,v1:172.21.15.97:6835/2090847706] up:boot 2023-10-08T09:53:32.573 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: fsmap 3 up:standby 2023-10-08T09:53:32.573 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:32 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:53:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2211539689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: Deploying daemon mds.d on smithi097 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.97:6834/2090847706,v1:172.21.15.97:6835/2090847706] up:boot 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: fsmap 3 up:standby 2023-10-08T09:53:32.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:32 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:53:33.203 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:32 smithi097 systemd[1]: Starting Ceph mds.d for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:33.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 -- 172.21.15.97:0/359400879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 msgr2=0x7fcb78101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/359400879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78101d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb680099c0 tx=0x7fcb68009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 -- 172.21.15.97:0/359400879 shutdown_connections 2023-10-08T09:53:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/359400879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb78102250 0x7fcb78108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/359400879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb781006c0 0x7fcb78100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/359400879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78101d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 -- 172.21.15.97:0/359400879 >> 172.21.15.97:0/359400879 conn(0x7fcb780fbc30 msgr2=0x7fcb780fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 -- 172.21.15.97:0/359400879 shutdown_connections 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.407+0000 7fcb7fdea700 1 -- 172.21.15.97:0/359400879 wait complete. 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 Processor -- start 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 -- start start 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb781006c0 0x7fcb78196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:33.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 0x7fcb7819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7819e130 con 0x7fcb781018b0 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7819e270 con 0x7fcb781006c0 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7819e400 con 0x7fcb78102250 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54860/0 (socket says 172.21.15.97:54860) 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 -- 172.21.15.97:0/1360280926 learned_addr learned my addr 172.21.15.97:0/1360280926 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:33.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7e387700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 0x7fcb7819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7db86700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb781006c0 0x7fcb78196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 -- 172.21.15.97:0/1360280926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 msgr2=0x7fcb7819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 0x7fcb7819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 -- 172.21.15.97:0/1360280926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb781006c0 msgr2=0x7fcb78196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb781006c0 0x7fcb78196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 -- 172.21.15.97:0/1360280926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680095f0 con 0x7fcb781018b0 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7e387700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 0x7fcb7819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7d385700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb680036f0 tx=0x7fcb680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6801d070 con 0x7fcb781018b0 2023-10-08T09:53:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb68004110 con 0x7fcb781018b0 2023-10-08T09:53:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68022e10 con 0x7fcb781018b0 2023-10-08T09:53:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.408+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7819e680 con 0x7fcb781018b0 2023-10-08T09:53:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.409+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7819eb30 con 0x7fcb781018b0 2023-10-08T09:53:33.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.409+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb78107b90 con 0x7fcb781018b0 2023-10-08T09:53:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.410+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcb68022470 con 0x7fcb781018b0 2023-10-08T09:53:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.411+0000 7fcb6effd700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 0x7fcb640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.411+0000 7fcb7db86700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 0x7fcb640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.411+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb680cdaf0 con 0x7fcb781018b0 2023-10-08T09:53:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.411+0000 7fcb7db86700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 0x7fcb640a39c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005950 tx=0x7fcb740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.415+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb6805d100 con 0x7fcb781018b0 2023-10-08T09:53:33.532 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:33 smithi097 podman[154955]: 2023-10-08T09:53:33.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.666+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb78067450 con 0x7fcb781018b0 2023-10-08T09:53:33.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.666+0000 7fcb6effd700 1 -- 172.21.15.97:0/1360280926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7fcb68097680 con 0x7fcb781018b0 2023-10-08T09:53:33.668 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:33.668 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 msgr2=0x7fcb640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 0x7fcb640a39c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005950 tx=0x7fcb740058e0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 msgr2=0x7fcb78196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb680036f0 tx=0x7fcb680047b0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 shutdown_connections 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb640a1500 0x7fcb640a39c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb78102250 0x7fcb7819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb781006c0 0x7fcb78196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 --2- 172.21.15.97:0/1360280926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb781018b0 0x7fcb78196b90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 >> 172.21.15.97:0/1360280926 conn(0x7fcb780fbc30 msgr2=0x7fcb78104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 shutdown_connections 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:33.669+0000 7fcb7fdea700 1 -- 172.21.15.97:0/1360280926 wait complete. 2023-10-08T09:53:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2023-10-08T09:53:33.952 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:33 smithi097 ceph-mds[155039]: -- 172.21.15.97:0/1288711292 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581c7c111e0 con 0x5581c7be7000 2023-10-08T09:53:33.953 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:33 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-d[154994]: starting mds.d at 2023-10-08T09:53:33.953 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:33 smithi097 bash[154955]: 5d6d68cbb50009e18494aca7ec3370993aee159241d171291df081fffc45373f 2023-10-08T09:53:33.953 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 09:53:33 smithi097 systemd[1]: Started Ceph mds.d for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:34.811 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.97:6836/3214411301,v1:172.21.15.97:6837/3214411301] up:boot 2023-10-08T09:53:34.811 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: fsmap 4 up:standby 2023-10-08T09:53:34.811 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:53:34.811 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1360280926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: Deploying daemon mds.e on smithi178 2023-10-08T09:53:34.812 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:34 smithi178 ceph-mon[103156]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.97:6836/3214411301,v1:172.21.15.97:6837/3214411301] up:boot 2023-10-08T09:53:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: fsmap 4 up:standby 2023-10-08T09:53:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:53:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1360280926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:34.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:34.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:34.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: Deploying daemon mds.e on smithi178 2023-10-08T09:53:34.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:34 smithi097 ceph-mon[99491]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.97:6836/3214411301,v1:172.21.15.97:6837/3214411301] up:boot 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: fsmap 4 up:standby 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1360280926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: Deploying daemon mds.e on smithi178 2023-10-08T09:53:35.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:34 smithi186 ceph-mon[104393]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:35.061 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 systemd[1]: Starting Ceph mds.e for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:35.623 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 podman[139643]: 2023-10-08T09:53:35.623 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 ceph-mds[139684]: -- 172.21.15.178:0/2571904495 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ed01f31e0 con 0x563ed01c8c00 2023-10-08T09:53:35.623 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: starting mds.e at 2023-10-08T09:53:35.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 -- 172.21.15.97:0/3885722463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c101120 msgr2=0x7f320c103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 --2- 172.21.15.97:0/3885722463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c101120 0x7f320c103510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009990 tx=0x7f31fc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 -- 172.21.15.97:0/3885722463 shutdown_connections 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 --2- 172.21.15.97:0/3885722463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c106380 0x7f320c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 --2- 172.21.15.97:0/3885722463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c101120 0x7f320c103510 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 --2- 172.21.15.97:0/3885722463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c103a50 0x7f320c105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 -- 172.21.15.97:0/3885722463 >> 172.21.15.97:0/3885722463 conn(0x7f320c0fab10 msgr2=0x7f320c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 -- 172.21.15.97:0/3885722463 shutdown_connections 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.822+0000 7f3214383700 1 -- 172.21.15.97:0/3885722463 wait complete. 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 Processor -- start 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 -- start start 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 0x7f320c1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 0x7f320c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19e0d0 con 0x7f320c106380 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19e210 con 0x7f320c101120 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3214383700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c19e3a0 con 0x7f320c103a50 2023-10-08T09:53:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54890/0 (socket says 172.21.15.97:54890) 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 -- 172.21.15.97:0/2848327449 learned_addr learned my addr 172.21.15.97:0/2848327449 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f321191e700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 0x7f320c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f321211f700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 0x7f320c1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 -- 172.21.15.97:0/2848327449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 msgr2=0x7f320c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 0x7f320c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 -- 172.21.15.97:0/2848327449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 msgr2=0x7f320c1965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 0x7f320c1965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 -- 172.21.15.97:0/2848327449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0095f0 con 0x7f320c106380 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f321191e700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 0x7f320c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:53:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f321211f700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 0x7f320c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.823+0000 7f3212920700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3204007ee0 tx=0x7f320400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.824+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204017070 con 0x7f320c106380 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.824+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320400f040 con 0x7f320c106380 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.824+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204005740 con 0x7f320c106380 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.824+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c19e680 con 0x7f320c106380 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.824+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c19eb40 con 0x7f320c106380 2023-10-08T09:53:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.825+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c0fc690 con 0x7f320c106380 2023-10-08T09:53:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.826+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f32040058a0 con 0x7f320c106380 2023-10-08T09:53:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.826+0000 7f32037fe700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 0x7f31f80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.826+0000 7f321211f700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 0x7f31f80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.826+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32040cc680 con 0x7f320c106380 2023-10-08T09:53:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.827+0000 7f321211f700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 0x7f31f80a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0037f0 tx=0x7f31fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:35.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:35.830+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3204096180 con 0x7f320c106380 2023-10-08T09:53:35.955 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 bash[139643]: dd8884e0d666e1c560670c68670674cf411ace12c8242538094b702bc3c672f6 2023-10-08T09:53:35.955 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 09:53:35 smithi178 systemd[1]: Started Ceph mds.e for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.066+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f320c067450 con 0x7f320c106380 2023-10-08T09:53:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.067+0000 7f32037fe700 1 -- 172.21.15.97:0/2848327449 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f32040997a0 con 0x7f320c106380 2023-10-08T09:53:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:36.068 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:53:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 msgr2=0x7f31f80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 0x7f31f80a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0037f0 tx=0x7f31fc005fb0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 msgr2=0x7f320c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3204007ee0 tx=0x7f320400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 shutdown_connections 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f31f80a1500 0x7f31f80a39c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f320c106380 0x7f320c19db00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f320c101120 0x7f320c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 --2- 172.21.15.97:0/2848327449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f320c103a50 0x7f320c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 >> 172.21.15.97:0/2848327449 conn(0x7f320c0fab10 msgr2=0x7f320c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 shutdown_connections 2023-10-08T09:53:36.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:36.069+0000 7f3214383700 1 -- 172.21.15.97:0/2848327449 wait complete. 2023-10-08T09:53:36.072 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: Deploying daemon mds.f on smithi186 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2848327449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6834/2717102390,v1:172.21.15.178:6835/2717102390] up:boot 2023-10-08T09:53:36.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: fsmap 5 up:standby 2023-10-08T09:53:36.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:36 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: Deploying daemon mds.f on smithi186 2023-10-08T09:53:36.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:36.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2848327449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:36.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6834/2717102390,v1:172.21.15.178:6835/2717102390] up:boot 2023-10-08T09:53:36.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: fsmap 5 up:standby 2023-10-08T09:53:36.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:36 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:53:37.009 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:37.009 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:37.009 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:37.009 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: Deploying daemon mds.f on smithi186 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2848327449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6834/2717102390,v1:172.21.15.178:6835/2717102390] up:boot 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: fsmap 5 up:standby 2023-10-08T09:53:37.010 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:36 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:53:37.275 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 systemd[1]: Starting Ceph mds.f for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:37.836 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 podman[142087]: 2023-10-08T09:53:37.836 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 ceph-mds[142127]: -- 172.21.15.186:0/3410094829 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573340151e0 con 0x557333fea800 2023-10-08T09:53:37.836 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 bash[142087]: 36708fdd089d3ef21d2ce5c10d1e0f81fdec111985e17b43791d96f35d28d6b4 2023-10-08T09:53:37.836 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: starting mds.f at 2023-10-08T09:53:38.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- 172.21.15.97:0/2512624965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 msgr2=0x7f5098101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 --2- 172.21.15.97:0/2512624965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098101d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50940099c0 tx=0x7f5094009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- 172.21.15.97:0/2512624965 shutdown_connections 2023-10-08T09:53:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 --2- 172.21.15.97:0/2512624965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5098102250 0x7f5098108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 --2- 172.21.15.97:0/2512624965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50981006c0 0x7f5098100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 --2- 172.21.15.97:0/2512624965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098101d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- 172.21.15.97:0/2512624965 >> 172.21.15.97:0/2512624965 conn(0x7f50980fbc30 msgr2=0x7f50980fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- 172.21.15.97:0/2512624965 shutdown_connections 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- 172.21.15.97:0/2512624965 wait complete. 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 Processor -- start 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.217+0000 7f509fdda700 1 -- start start 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50981006c0 0x7f50981965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5098102250 0x7f509819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e0d0 con 0x7f50981018b0 2023-10-08T09:53:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e210 con 0x7f5098102250 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509819e3a0 con 0x7f50981006c0 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54898/0 (socket says 172.21.15.97:54898) 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 -- 172.21.15.97:0/1407964531 learned_addr learned my addr 172.21.15.97:0/1407964531 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509db76700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50981006c0 0x7f50981965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 -- 172.21.15.97:0/1407964531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50981006c0 msgr2=0x7f50981965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509e377700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5098102250 0x7f509819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50981006c0 0x7f50981965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 -- 172.21.15.97:0/1407964531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5098102250 msgr2=0x7f509819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5098102250 0x7f509819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 -- 172.21.15.97:0/1407964531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940095f0 con 0x7f50981018b0 2023-10-08T09:53:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509d375700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f50940036f0 tx=0x7f5094003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509401d070 con 0x7f50981018b0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5094022470 con 0x7f50981018b0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509819e620 con 0x7f50981018b0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400f6e0 con 0x7f50981018b0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.218+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509819ead0 con 0x7f50981018b0 2023-10-08T09:53:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.219+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5098107a90 con 0x7f50981018b0 2023-10-08T09:53:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.220+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5094022890 con 0x7f50981018b0 2023-10-08T09:53:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.221+0000 7f508effd700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 0x7f50840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.221+0000 7f509db76700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 0x7f50840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.221+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50940cde90 con 0x7f50981018b0 2023-10-08T09:53:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.221+0000 7f509db76700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 0x7f50840a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5088005fd0 tx=0x7f5088005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.224+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5094097a20 con 0x7f50981018b0 2023-10-08T09:53:38.275 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 09:53:37 smithi186 systemd[1]: Started Ceph mds.f for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.464+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5098067450 con 0x7f50981018b0 2023-10-08T09:53:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.465+0000 7f508effd700 1 -- 172.21.15.97:0/1407964531 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f509409b040 con 0x7f50981018b0 2023-10-08T09:53:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:38.466 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 msgr2=0x7f50840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 0x7f50840a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5088005fd0 tx=0x7f5088005e50 comp rx=0 tx=0).stop 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 msgr2=0x7f5098196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f50940036f0 tx=0x7f5094003970 comp rx=0 tx=0).stop 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 shutdown_connections 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f50840a1480 0x7f50840a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5098102250 0x7f509819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50981006c0 0x7f50981965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 --2- 172.21.15.97:0/1407964531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50981018b0 0x7f5098196ae0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 >> 172.21.15.97:0/1407964531 conn(0x7f50980fbc30 msgr2=0x7f5098104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 shutdown_connections 2023-10-08T09:53:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:38.467+0000 7f509fdda700 1 -- 172.21.15.97:0/1407964531 wait complete. 2023-10-08T09:53:38.469 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2023-10-08T09:53:39.183 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6834/3587378302,v1:172.21.15.186:6835/3587378302] up:boot 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: fsmap 6 up:standby 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: Deploying daemon mds.c on smithi186 2023-10-08T09:53:39.184 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1407964531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:39.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6834/3587378302,v1:172.21.15.186:6835/3587378302] up:boot 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: fsmap 6 up:standby 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: Deploying daemon mds.c on smithi186 2023-10-08T09:53:39.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1407964531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6834/3587378302,v1:172.21.15.186:6835/3587378302] up:boot 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: fsmap 6 up:standby 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:39.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:39.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:39.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: Deploying daemon mds.c on smithi186 2023-10-08T09:53:39.206 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1407964531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:39.525 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:39 smithi186 systemd[1]: Starting Ceph mds.c for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:39.980 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:39 smithi186 podman[142429]: 2023-10-08T09:53:39.980 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:39 smithi186 ceph-mds[142469]: -- 172.21.15.186:0/3014701722 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563dcce291e0 con 0x563dccdff000 2023-10-08T09:53:39.980 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:39 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: starting mds.c at 2023-10-08T09:53:40.275 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:39 smithi186 bash[142429]: 8287837bdc2c9ad98857fe5494c1970f3f35da80fdc4ea4e1de123140b0336cd 2023-10-08T09:53:40.275 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 09:53:40 smithi186 systemd[1]: Started Ceph mds.c for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:40.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- 172.21.15.97:0/1161128014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8641023a0 msgr2=0x7fb864108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 --2- 172.21.15.97:0/1161128014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8641023a0 0x7fb864108c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b0a0 tx=0x7fb85c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- 172.21.15.97:0/1161128014 shutdown_connections 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 --2- 172.21.15.97:0/1161128014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8641023a0 0x7fb864108c50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 --2- 172.21.15.97:0/1161128014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8640fe990 0x7fb8640fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 --2- 172.21.15.97:0/1161128014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb864101a00 0x7fb864101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- 172.21.15.97:0/1161128014 >> 172.21.15.97:0/1161128014 conn(0x7fb8640fa2c0 msgr2=0x7fb8640fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- 172.21.15.97:0/1161128014 shutdown_connections 2023-10-08T09:53:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- 172.21.15.97:0/1161128014 wait complete. 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 Processor -- start 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.617+0000 7fb86c709700 1 -- start start 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb864101a00 0x7fb864196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8641023a0 0x7fb86419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86419e240 con 0x7fb8640fe990 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86419e380 con 0x7fb864101a00 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86419e510 con 0x7fb8641023a0 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54908/0 (socket says 172.21.15.97:54908) 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 -- 172.21.15.97:0/751882230 learned_addr learned my addr 172.21.15.97:0/751882230 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb869ca4700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb864101a00 0x7fb864196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86aca6700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8641023a0 0x7fb86419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 -- 172.21.15.97:0/751882230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8641023a0 msgr2=0x7fb86419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8641023a0 0x7fb86419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 -- 172.21.15.97:0/751882230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb864101a00 msgr2=0x7fb864196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb864101a00 0x7fb864196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 -- 172.21.15.97:0/751882230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c009d00 con 0x7fb8640fe990 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86a4a5700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e970 tx=0x7fb85400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854009790 con 0x7fb8640fe990 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86419e7f0 con 0x7fb8640fe990 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb85400c7e0 con 0x7fb8640fe990 2023-10-08T09:53:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854003c70 con 0x7fb8640fe990 2023-10-08T09:53:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.618+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86419ed00 con 0x7fb8640fe990 2023-10-08T09:53:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.620+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb864107bc0 con 0x7fb8640fe990 2023-10-08T09:53:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.620+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb85400cc50 con 0x7fb8640fe990 2023-10-08T09:53:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.621+0000 7fb85b7fe700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 0x7fb8500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.621+0000 7fb869ca4700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 0x7fb8500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.621+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb854014070 con 0x7fb8640fe990 2023-10-08T09:53:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.621+0000 7fb869ca4700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 0x7fb8500a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb860005950 tx=0x7fb86000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.625+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb854096550 con 0x7fb8640fe990 2023-10-08T09:53:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.857+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb864066bc0 con 0x7fb8640fe990 2023-10-08T09:53:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.858+0000 7fb85b7fe700 1 -- 172.21.15.97:0/751882230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fb854003dd0 con 0x7fb8640fe990 2023-10-08T09:53:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:40.859 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 msgr2=0x7fb8500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 0x7fb8500a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb860005950 tx=0x7fb86000b410 comp rx=0 tx=0).stop 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 msgr2=0x7fb864196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e970 tx=0x7fb85400ec80 comp rx=0 tx=0).stop 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 shutdown_connections 2023-10-08T09:53:40.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8500a1480 0x7fb8500a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8641023a0 0x7fb86419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fe990 0x7fb864196710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 --2- 172.21.15.97:0/751882230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb864101a00 0x7fb864196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 >> 172.21.15.97:0/751882230 conn(0x7fb8640fa2c0 msgr2=0x7fb864104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 shutdown_connections 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:40.860+0000 7fb86c709700 1 -- 172.21.15.97:0/751882230 wait complete. 2023-10-08T09:53:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: Deploying daemon mds.h on smithi178 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/751882230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:41.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:boot 2023-10-08T09:53:41.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: fsmap 7 up:standby 2023-10-08T09:53:41.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:53:41.384 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:41.384 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: Deploying daemon mds.h on smithi178 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/751882230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:boot 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: fsmap 7 up:standby 2023-10-08T09:53:41.385 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:53:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: Deploying daemon mds.h on smithi178 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/751882230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:boot 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: fsmap 7 up:standby 2023-10-08T09:53:41.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:53:41.705 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:41 smithi178 systemd[1]: Starting Ceph mds.h for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:42.017 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:41 smithi178 podman[139981]: 2023-10-08T09:53:42.017 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-mds[140022]: -- 172.21.15.178:0/2126434402 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56106d38b1e0 con 0x56106d361000 2023-10-08T09:53:42.018 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:41 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: starting mds.h at 2023-10-08T09:53:42.455 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:42 smithi178 bash[139981]: e6ebdff16e14da1a12a6232d3e9548c41b11f69bee4a89916d94cd27caa445fc 2023-10-08T09:53:42.455 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 09:53:42 smithi178 systemd[1]: Started Ceph mds.h for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:42.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 -- 172.21.15.97:0/3863594544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48740ff5a0 msgr2=0x7f48740ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 --2- 172.21.15.97:0/3863594544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48740ff5a0 0x7f48740ff9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48640099c0 tx=0x7f4864009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 -- 172.21.15.97:0/3863594544 shutdown_connections 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 --2- 172.21.15.97:0/3863594544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874101130 0x7f48741079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 --2- 172.21.15.97:0/3863594544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48740ff5a0 0x7f48740ff9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 --2- 172.21.15.97:0/3863594544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4874100790 0x7f4874100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 -- 172.21.15.97:0/3863594544 >> 172.21.15.97:0/3863594544 conn(0x7f48740fab10 msgr2=0x7f48740fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 -- 172.21.15.97:0/3863594544 shutdown_connections 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.008+0000 7f487a76e700 1 -- 172.21.15.97:0/3863594544 wait complete. 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 Processor -- start 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 -- start start 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740ff5a0 0x7f48741965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874100790 0x7f4874196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419e0d0 con 0x7f4874101130 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419e210 con 0x7f48740ff5a0 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f487a76e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487419e3a0 con 0x7f4874100790 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:43.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f48737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874100790 0x7f4874196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54926/0 (socket says 172.21.15.97:54926) 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 -- 172.21.15.97:0/2588254463 learned_addr learned my addr 172.21.15.97:0/2588254463 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4873fff700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740ff5a0 0x7f48741965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 -- 172.21.15.97:0/2588254463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874100790 msgr2=0x7f4874196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874100790 0x7f4874196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 -- 172.21.15.97:0/2588254463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740ff5a0 msgr2=0x7f48741965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740ff5a0 0x7f48741965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.009+0000 7f4878d0b700 1 -- 172.21.15.97:0/2588254463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640095f0 con 0x7f4874101130 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f4878d0b700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f486c00ef30 tx=0x7f486c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c018070 con 0x7f4874101130 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f486c00f040 con 0x7f4874101130 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00cd00 con 0x7f4874101130 2023-10-08T09:53:43.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487419e620 con 0x7f4874101130 2023-10-08T09:53:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.010+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487419eae0 con 0x7f4874101130 2023-10-08T09:53:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.011+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874067450 con 0x7f4874101130 2023-10-08T09:53:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.012+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f486c003a00 con 0x7f4874101130 2023-10-08T09:53:43.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.012+0000 7f48717fa700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 0x7f485c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:43.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.012+0000 7f4873fff700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 0x7f485c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:43.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.012+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f486c0ccd60 con 0x7f4874101130 2023-10-08T09:53:43.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.013+0000 7f4873fff700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 0x7f485c0a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4864003730 tx=0x7f4864000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:43.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.016+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f486c0968f0 con 0x7f4874101130 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6836/2015632800,v1:172.21.15.178:6837/2015632800] up:boot 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: fsmap 8 up:standby 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:43.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:43.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:43.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:43 smithi097 ceph-mon[99491]: Deploying daemon mds.i on smithi186 2023-10-08T09:53:43.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.256+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f487410adc0 con 0x7f4874101130 2023-10-08T09:53:43.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.257+0000 7f48717fa700 1 -- 172.21.15.97:0/2588254463 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7f486c099f10 con 0x7f4874101130 2023-10-08T09:53:43.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:43.258 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 msgr2=0x7f485c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 0x7f485c0a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4864003730 tx=0x7f4864000bc0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 msgr2=0x7f487419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f486c00ef30 tx=0x7f486c00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 shutdown_connections 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f485c0a1480 0x7f485c0a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4874101130 0x7f487419db00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740ff5a0 0x7f48741965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 --2- 172.21.15.97:0/2588254463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4874100790 0x7f4874196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:43.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 >> 172.21.15.97:0/2588254463 conn(0x7f48740fab10 msgr2=0x7f48740fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:43.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 shutdown_connections 2023-10-08T09:53:43.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:43.259+0000 7f487a76e700 1 -- 172.21.15.97:0/2588254463 wait complete. 2023-10-08T09:53:43.261 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2023-10-08T09:53:43.390 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:43.390 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6836/2015632800,v1:172.21.15.178:6837/2015632800] up:boot 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: fsmap 8 up:standby 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:43.391 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:43 smithi186 ceph-mon[104393]: Deploying daemon mds.i on smithi186 2023-10-08T09:53:43.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:43.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6836/2015632800,v1:172.21.15.178:6837/2015632800] up:boot 2023-10-08T09:53:43.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: fsmap 8 up:standby 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:43.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:43 smithi178 ceph-mon[103156]: Deploying daemon mds.i on smithi186 2023-10-08T09:53:43.775 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:43 smithi186 systemd[1]: Starting Ceph mds.i for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:44.170 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:44 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2588254463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:44.171 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:43 smithi186 podman[142770]: 2023-10-08T09:53:44.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:44 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2588254463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:44.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:44 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2588254463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:44.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:44 smithi186 ceph-mds[142810]: -- 172.21.15.186:0/3327331550 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564cb645f1e0 con 0x564cb6434c00 2023-10-08T09:53:44.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:44 smithi186 bash[142770]: bb9adb0f483220985f65df76c9d8d443eb0a0881a84c9f6c43aee0d62bbbdf09 2023-10-08T09:53:44.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:44 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: starting mds.i at 2023-10-08T09:53:44.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 09:53:44 smithi186 systemd[1]: Started Ceph mds.i for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:45.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:45.302 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:45.303 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:45 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 -- 172.21.15.97:0/4108799441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2824102250 msgr2=0x7f2824108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 --2- 172.21.15.97:0/4108799441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2824102250 0x7f2824108b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b0d0 tx=0x7f281c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 -- 172.21.15.97:0/4108799441 shutdown_connections 2023-10-08T09:53:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 --2- 172.21.15.97:0/4108799441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2824102250 0x7f2824108b00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 --2- 172.21.15.97:0/4108799441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28241006c0 0x7f2824100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 --2- 172.21.15.97:0/4108799441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 -- 172.21.15.97:0/4108799441 >> 172.21.15.97:0/4108799441 conn(0x7f28240fbc30 msgr2=0x7f28240fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:45.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 -- 172.21.15.97:0/4108799441 shutdown_connections 2023-10-08T09:53:45.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.404+0000 7f282a1ca700 1 -- 172.21.15.97:0/4108799441 wait complete. 2023-10-08T09:53:45.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 Processor -- start 2023-10-08T09:53:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 -- start start 2023-10-08T09:53:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102250 0x7f282419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e2e0 con 0x7f28241006c0 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e420 con 0x7f28241018b0 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f282a1ca700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282419e5b0 con 0x7f2824102250 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54934/0 (socket says 172.21.15.97:54934) 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 -- 172.21.15.97:0/2544255842 learned_addr learned my addr 172.21.15.97:0/2544255842 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f2822ffd700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 -- 172.21.15.97:0/2544255842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102250 msgr2=0x7f282419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102250 0x7f282419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 -- 172.21.15.97:0/2544255842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 msgr2=0x7f2824196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 -- 172.21.15.97:0/2544255842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009d00 con 0x7f28241006c0 2023-10-08T09:53:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f2822ffd700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.405+0000 7f28237fe700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f281800b500 tx=0x7f281800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.406+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818011830 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.406+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2818011e50 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.406+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282419e890 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.406+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800f2e0 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.406+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282419ee70 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.407+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2824107a90 con 0x7f28241006c0 2023-10-08T09:53:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.408+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2818010580 con 0x7f28241006c0 2023-10-08T09:53:45.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.408+0000 7f2820ff9700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:45.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.408+0000 7f2822ffd700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:45.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.408+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28180cc840 con 0x7f28241006c0 2023-10-08T09:53:45.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.409+0000 7f2822ffd700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 0x7f280c0a39c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2814005950 tx=0x7f2814005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:45.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.412+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2818096340 con 0x7f28241006c0 2023-10-08T09:53:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:45.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:45.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:45 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:45 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.657+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2824067450 con 0x7f28241006c0 2023-10-08T09:53:45.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.658+0000 7f2820ff9700 1 -- 172.21.15.97:0/2544255842 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7f2818099960 con 0x7f28241006c0 2023-10-08T09:53:45.659 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:45.659 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 msgr2=0x7f280c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 0x7f280c0a39c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2814005950 tx=0x7f2814005f50 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 msgr2=0x7f28241967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f281800b500 tx=0x7f281800b8c0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 shutdown_connections 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102250 0x7f282419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28241006c0 0x7f28241967b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 --2- 172.21.15.97:0/2544255842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28241018b0 0x7f2824196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 >> 172.21.15.97:0/2544255842 conn(0x7f28240fbc30 msgr2=0x7f2824104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 shutdown_connections 2023-10-08T09:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:45.660+0000 7f282a1ca700 1 -- 172.21.15.97:0/2544255842 wait complete. 2023-10-08T09:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2023-10-08T09:53:46.202 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 systemd[1]: Starting Ceph mds.g for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:46.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mon[99491]: Deploying daemon mds.g on smithi097 2023-10-08T09:53:46.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:boot 2023-10-08T09:53:46.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mon[99491]: fsmap 9 up:standby 2023-10-08T09:53:46.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:53:46.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2544255842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:46.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:46 smithi178 ceph-mon[103156]: Deploying daemon mds.g on smithi097 2023-10-08T09:53:46.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:46 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:boot 2023-10-08T09:53:46.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:46 smithi178 ceph-mon[103156]: fsmap 9 up:standby 2023-10-08T09:53:46.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:53:46.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:46 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2544255842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:46 smithi186 ceph-mon[104393]: Deploying daemon mds.g on smithi097 2023-10-08T09:53:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:46 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:boot 2023-10-08T09:53:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:46 smithi186 ceph-mon[104393]: fsmap 9 up:standby 2023-10-08T09:53:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:53:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:46 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2544255842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:46.552 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 podman[155578]: 2023-10-08T09:53:46.953 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-mds[155618]: -- 172.21.15.97:0/865870489 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e0d83211e0 con 0x55e0d82f6c00 2023-10-08T09:53:46.953 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-g[155594]: starting mds.g at 2023-10-08T09:53:46.954 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 bash[155578]: d49f20626efd05c380185d6041fc5bb2cf187d69501cc31049ea41eab9eef3db 2023-10-08T09:53:46.954 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 09:53:46 smithi097 systemd[1]: Started Ceph mds.g for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:47.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:47.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:47.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:47.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 -- 172.21.15.97:0/1528636244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100170 msgr2=0x7f5ca0107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/1528636244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100170 0x7f5ca0107a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b0a0 tx=0x7f5c9800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 -- 172.21.15.97:0/1528636244 shutdown_connections 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/1528636244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100170 0x7f5ca0107a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/1528636244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100b60 0x7f5ca0100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/1528636244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 0x7f5ca00ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 -- 172.21.15.97:0/1528636244 >> 172.21.15.97:0/1528636244 conn(0x7f5ca00fa2c0 msgr2=0x7f5ca00fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 -- 172.21.15.97:0/1528636244 shutdown_connections 2023-10-08T09:53:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.800+0000 7f5ca6b69700 1 -- 172.21.15.97:0/1528636244 wait complete. 2023-10-08T09:53:47.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 Processor -- start 2023-10-08T09:53:47.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 -- start start 2023-10-08T09:53:47.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 0x7f5ca01035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:47.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100170 0x7f5ca0101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:47.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0102950 con 0x7f5ca0100b60 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0102ac0 con 0x7f5ca0100170 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca6b69700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01a0770 con 0x7f5ca00ff7b0 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca5106700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca5106700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54966/0 (socket says 172.21.15.97:54966) 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.801+0000 7f5ca5106700 1 -- 172.21.15.97:0/4007585735 learned_addr learned my addr 172.21.15.97:0/4007585735 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca4905700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 0x7f5ca01035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5c9ffff700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100170 0x7f5ca0101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 -- 172.21.15.97:0/4007585735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 msgr2=0x7f5ca01035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 0x7f5ca01035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 -- 172.21.15.97:0/4007585735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100170 msgr2=0x7f5ca0101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:47.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100170 0x7f5ca0101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 -- 172.21.15.97:0/4007585735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009d00 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca5106700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006070 tx=0x7f5c98004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800e040 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9801db50 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca01a09a0 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9801bc00 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.802+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca01a0dc0 con 0x7f5ca0100b60 2023-10-08T09:53:47.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.803+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca0066bc0 con 0x7f5ca0100b60 2023-10-08T09:53:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.804+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5c9801dcc0 con 0x7f5ca0100b60 2023-10-08T09:53:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.805+0000 7f5c9dffb700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 0x7f5c900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.805+0000 7f5ca4905700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 0x7f5c900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.805+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c980ceb00 con 0x7f5ca0100b60 2023-10-08T09:53:47.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.805+0000 7f5ca4905700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 0x7f5c900a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005950 tx=0x7f5c9400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:47.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:47.808+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5c98098690 con 0x7f5ca0100b60 2023-10-08T09:53:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.070+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ca010afb0 con 0x7f5ca0100b60 2023-10-08T09:53:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.070+0000 7f5c9dffb700 1 -- 172.21.15.97:0/4007585735 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7f5c9809bcb0 con 0x7f5ca0100b60 2023-10-08T09:53:48.072 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:48.072 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:53:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.073+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 msgr2=0x7f5c900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.073+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 0x7f5c900a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005950 tx=0x7f5c9400b410 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.073+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 msgr2=0x7f5ca01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.073+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006070 tx=0x7f5c98004d00 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 shutdown_connections 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5c900a1480 0x7f5c900a3940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ca0100b60 0x7f5ca01021b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ca00ff7b0 0x7f5ca01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 --2- 172.21.15.97:0/4007585735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ca0100170 0x7f5ca0101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 >> 172.21.15.97:0/4007585735 conn(0x7f5ca00fa2c0 msgr2=0x7f5ca00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 shutdown_connections 2023-10-08T09:53:48.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:48.074+0000 7f5ca6b69700 1 -- 172.21.15.97:0/4007585735 wait complete. 2023-10-08T09:53:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2023-10-08T09:53:48.284 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mon[99491]: Deploying daemon mds.a on smithi097 2023-10-08T09:53:48.284 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.97:6838/1211581786,v1:172.21.15.97:6839/1211581786] up:boot 2023-10-08T09:53:48.284 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mon[99491]: fsmap 10 up:standby 2023-10-08T09:53:48.284 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:53:48.285 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4007585735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:48.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:48 smithi178 ceph-mon[103156]: Deploying daemon mds.a on smithi097 2023-10-08T09:53:48.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:48 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.97:6838/1211581786,v1:172.21.15.97:6839/1211581786] up:boot 2023-10-08T09:53:48.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:48 smithi178 ceph-mon[103156]: fsmap 10 up:standby 2023-10-08T09:53:48.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:48 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:53:48.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:48 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4007585735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:48 smithi186 ceph-mon[104393]: Deploying daemon mds.a on smithi097 2023-10-08T09:53:48.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:48 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.97:6838/1211581786,v1:172.21.15.97:6839/1211581786] up:boot 2023-10-08T09:53:48.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:48 smithi186 ceph-mon[104393]: fsmap 10 up:standby 2023-10-08T09:53:48.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:48 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:53:48.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:48 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4007585735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:48.550 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 systemd[1]: Starting Ceph mds.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:48.801 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 podman[155962]: 2023-10-08T09:53:49.148 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-mds[156002]: -- 172.21.15.97:0/2013906764 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dbf3d2f1e0 con 0x55dbf3d04800 2023-10-08T09:53:49.148 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-a[155978]: starting mds.a at 2023-10-08T09:53:49.148 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 bash[155962]: 46fea3fdc95ec6221374c7770057f6b5cc739a7684eb94648df856196738ab3c 2023-10-08T09:53:49.148 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 09:53:48 smithi097 systemd[1]: Started Ceph mds.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:49.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:49 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:49.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:49 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:53:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:49 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:50.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:50.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:50 smithi178 ceph-mon[103156]: Deploying daemon mds.b on smithi178 2023-10-08T09:53:50.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:50 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.97:6840/3143185366,v1:172.21.15.97:6841/3143185366] up:boot 2023-10-08T09:53:50.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:50 smithi178 ceph-mon[103156]: fsmap 11 up:standby 2023-10-08T09:53:50.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:50 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:53:50.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 -- 172.21.15.97:0/73368912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a00ff850 msgr2=0x7fa6a00ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:50.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 --2- 172.21.15.97:0/73368912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a00ff850 0x7fa6a00ffcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa690009990 tx=0x7fa690009ca0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 -- 172.21.15.97:0/73368912 shutdown_connections 2023-10-08T09:53:50.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 --2- 172.21.15.97:0/73368912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 --2- 172.21.15.97:0/73368912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a0100bb0 0x7fa6a0100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 --2- 172.21.15.97:0/73368912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a00ff850 0x7fa6a00ffcb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 -- 172.21.15.97:0/73368912 >> 172.21.15.97:0/73368912 conn(0x7fa6a00fa2c0 msgr2=0x7fa6a00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:50.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 -- 172.21.15.97:0/73368912 shutdown_connections 2023-10-08T09:53:50.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.231+0000 7fa6a5391700 1 -- 172.21.15.97:0/73368912 wait complete. 2023-10-08T09:53:50.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 Processor -- start 2023-10-08T09:53:50.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 -- start start 2023-10-08T09:53:50.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff850 0x7fa6a0195510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:50.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:50.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d040 con 0x7fa6a0100bb0 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d180 con 0x7fa6a01001f0 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa6a5391700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a019d310 con 0x7fa6a00ff850 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa69f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa69e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa69effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff850 0x7fa6a0195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.232+0000 7fa69f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54994/0 (socket says 172.21.15.97:54994) 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:34962/0 (socket says 172.21.15.97:34962) 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 -- 172.21.15.97:0/3463531270 learned_addr learned my addr 172.21.15.97:0/3463531270 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 -- 172.21.15.97:0/3463531270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff850 msgr2=0x7fa6a0195510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff850 0x7fa6a0195510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 -- 172.21.15.97:0/3463531270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 msgr2=0x7fa6a0195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:50.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 -- 172.21.15.97:0/3463531270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900095f0 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa69f7fe700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa69800ef30 tx=0x7fa69800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698018070 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a00fe740 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69800f040 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69800cd00 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.233+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a00fec50 con 0x7fa6a0100bb0 2023-10-08T09:53:50.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.234+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a00678a0 con 0x7fa6a0100bb0 2023-10-08T09:53:50.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.235+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa698003a00 con 0x7fa6a0100bb0 2023-10-08T09:53:50.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.236+0000 7fa687fff700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 0x7fa6880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:50.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.236+0000 7fa69effd700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 0x7fa6880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:50.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.236+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6980cce00 con 0x7fa6a0100bb0 2023-10-08T09:53:50.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.236+0000 7fa69effd700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 0x7fa6880a39c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0068a10 tx=0x7fa694007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:50.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.239+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa6980968f0 con 0x7fa6a0100bb0 2023-10-08T09:53:50.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:50 smithi097 ceph-mon[99491]: Deploying daemon mds.b on smithi178 2023-10-08T09:53:50.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:50 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.97:6840/3143185366,v1:172.21.15.97:6841/3143185366] up:boot 2023-10-08T09:53:50.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:50 smithi097 ceph-mon[99491]: fsmap 11 up:standby 2023-10-08T09:53:50.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:50 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:53:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.479+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a010b0d0 con 0x7fa6a0100bb0 2023-10-08T09:53:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.479+0000 7fa687fff700 1 -- 172.21.15.97:0/3463531270 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9260 (secure 0 0 0) 0x7fa698099f10 con 0x7fa6a0100bb0 2023-10-08T09:53:50.481 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:50.481 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:53:50.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 msgr2=0x7fa6880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:50.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 0x7fa6880a39c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0068a10 tx=0x7fa694007400 comp rx=0 tx=0).stop 2023-10-08T09:53:50.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 msgr2=0x7fa6a019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:50.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa69800ef30 tx=0x7fa69800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 shutdown_connections 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6880a1500 0x7fa6880a39c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a0100bb0 0x7fa6a019ca70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6a00ff850 0x7fa6a0195510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 --2- 172.21.15.97:0/3463531270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6a01001f0 0x7fa6a0195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 >> 172.21.15.97:0/3463531270 conn(0x7fa6a00fa2c0 msgr2=0x7fa6a0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 shutdown_connections 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:50.482+0000 7fa6a5391700 1 -- 172.21.15.97:0/3463531270 wait complete. 2023-10-08T09:53:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2023-10-08T09:53:50.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:50 smithi186 ceph-mon[104393]: Deploying daemon mds.b on smithi178 2023-10-08T09:53:50.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:50 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.97:6840/3143185366,v1:172.21.15.97:6841/3143185366] up:boot 2023-10-08T09:53:50.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:50 smithi186 ceph-mon[104393]: fsmap 11 up:standby 2023-10-08T09:53:50.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:50 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:53:50.705 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:50 smithi178 systemd[1]: Starting Ceph mds.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T09:53:51.051 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:50 smithi178 podman[140322]: 2023-10-08T09:53:51.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:51 smithi097 ceph-mon[99491]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:51 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3463531270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:51 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mon[103156]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3463531270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-mds[140362]: -- 172.21.15.178:0/497058066 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561a7c9bb1e0 con 0x561a7c991000 2023-10-08T09:53:51.456 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:51 smithi178 bash[140322]: 9c00c6a5696f92ef292e419c3d169c69ed2e492d30eac70dbeee7c9a3475c9a5 2023-10-08T09:53:51.456 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:51 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: starting mds.b at 2023-10-08T09:53:51.456 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:53:51 smithi178 systemd[1]: Started Ceph mds.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T09:53:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:51 smithi186 ceph-mon[104393]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:51 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3463531270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:51 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:boot 2023-10-08T09:53:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: fsmap 12 up:standby 2023-10-08T09:53:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:53:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:52 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:boot 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: fsmap 12 up:standby 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:52 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:52.497 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:boot 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: fsmap 12 up:standby 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:52 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 -- 172.21.15.97:0/4235986549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c100170 msgr2=0x7fdb5c107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 --2- 172.21.15.97:0/4235986549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c100170 0x7fdb5c107a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b0a0 tx=0x7fdb5400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 -- 172.21.15.97:0/4235986549 shutdown_connections 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 --2- 172.21.15.97:0/4235986549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c100170 0x7fdb5c107a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 --2- 172.21.15.97:0/4235986549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100b60 0x7fdb5c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 --2- 172.21.15.97:0/4235986549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 -- 172.21.15.97:0/4235986549 >> 172.21.15.97:0/4235986549 conn(0x7fdb5c0fa2c0 msgr2=0x7fdb5c0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 -- 172.21.15.97:0/4235986549 shutdown_connections 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.622+0000 7fdb641d9700 1 -- 172.21.15.97:0/4235986549 wait complete. 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 Processor -- start 2023-10-08T09:53:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 -- start start 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c100b60 0x7fdb5c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19e1b0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19e2f0 con 0x7fdb5c100b60 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb641d9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19e480 con 0x7fdb5c100170 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb61f75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb61774700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb61774700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:56706/0 (socket says 172.21.15.97:56706) 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb61774700 1 -- 172.21.15.97:0/1797212049 learned_addr learned my addr 172.21.15.97:0/1797212049 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb61f75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55004/0 (socket says 172.21.15.97:55004) 2023-10-08T09:53:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.623+0000 7fdb62776700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c100b60 0x7fdb5c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 -- 172.21.15.97:0/1797212049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 msgr2=0x7fdb5c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 -- 172.21.15.97:0/1797212049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c100b60 msgr2=0x7fdb5c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c100b60 0x7fdb5c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 -- 172.21.15.97:0/1797212049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb54009d00 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61774700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb61f75700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d950 tx=0x7fdb4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f890 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4c010460 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c19e760 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00e610 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.624+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c19ec70 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.625+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c066bc0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.626+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdb4c0105d0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.626+0000 7fdb52ffd700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 0x7fdb480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.626+0000 7fdb61774700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 0x7fdb480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.627+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdb4c0cc9a0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.627+0000 7fdb61774700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 0x7fdb480a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005950 tx=0x7fdb5800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.630+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb4c0964a0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.875+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb5c10adb0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.876+0000 7fdb52ffd700 1 -- 172.21.15.97:0/1797212049 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10055 (secure 0 0 0) 0x7fdb4c099ac0 con 0x7fdb5c0ff7b0 2023-10-08T09:53:52.878 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:52.878 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-08T09:53:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 msgr2=0x7fdb480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 0x7fdb480a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005950 tx=0x7fdb5800b500 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 msgr2=0x7fdb5c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d950 tx=0x7fdb4c00dc60 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 shutdown_connections 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdb480a1480 0x7fdb480a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb5c100b60 0x7fdb5c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb5c0ff7b0 0x7fdb5c196680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 --2- 172.21.15.97:0/1797212049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb5c100170 0x7fdb5c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 >> 172.21.15.97:0/1797212049 conn(0x7fdb5c0fa2c0 msgr2=0x7fdb5c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 shutdown_connections 2023-10-08T09:53:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:52.878+0000 7fdb641d9700 1 -- 172.21.15.97:0/1797212049 wait complete. 2023-10-08T09:53:52.882 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 13 2023-10-08T09:53:52.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:53:53.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 -- 172.21.15.97:0/1495442399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41006c0 msgr2=0x7f87f4100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 --2- 172.21.15.97:0/1495442399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41006c0 0x7f87f4100ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87e40099c0 tx=0x7f87e4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 -- 172.21.15.97:0/1495442399 shutdown_connections 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 --2- 172.21.15.97:0/1495442399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f4102250 0x7f87f4108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 --2- 172.21.15.97:0/1495442399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41006c0 0x7f87f4100ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 --2- 172.21.15.97:0/1495442399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41018b0 0x7f87f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 -- 172.21.15.97:0/1495442399 >> 172.21.15.97:0/1495442399 conn(0x7f87f40fbc50 msgr2=0x7f87f40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 -- 172.21.15.97:0/1495442399 shutdown_connections 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.017+0000 7f87f90d5700 1 -- 172.21.15.97:0/1495442399 wait complete. 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 Processor -- start 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 -- start start 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f41006c0 0x7f87f4196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f4102250 0x7f87f419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419e150 con 0x7f87f41018b0 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419e290 con 0x7f87f41006c0 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.018+0000 7f87f90d5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419e420 con 0x7f87f4102250 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55020/0 (socket says 172.21.15.97:55020) 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 -- 172.21.15.97:0/4123298616 learned_addr learned my addr 172.21.15.97:0/4123298616 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f2d9d700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f41006c0 0x7f87f4196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f359e700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f4102250 0x7f87f419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 -- 172.21.15.97:0/4123298616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f4102250 msgr2=0x7f87f419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f4102250 0x7f87f419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 -- 172.21.15.97:0/4123298616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f41006c0 msgr2=0x7f87f4196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f41006c0 0x7f87f4196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 -- 172.21.15.97:0/4123298616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40095f0 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f259c700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87e800e9a0 tx=0x7f87e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8009790 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f419e700 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e8010430 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8018680 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.019+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f419ec60 con 0x7f87f41018b0 2023-10-08T09:53:53.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.020+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f4107ae0 con 0x7f87f41018b0 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.022+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f87e80105a0 con 0x7f87f41018b0 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.023+0000 7f87dbfff700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 0x7f87dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.023+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87e8014070 con 0x7f87f41018b0 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.023+0000 7f87f2d9d700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 0x7f87dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.023+0000 7f87f2d9d700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 0x7f87dc0a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87e4003730 tx=0x7f87e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.026+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87e8096700 con 0x7f87f41018b0 2023-10-08T09:53:53.169 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:53 smithi097 ceph-mon[99491]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:53.169 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:53 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1797212049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.232+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87f410bff0 con 0x7f87f41018b0 2023-10-08T09:53:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.233+0000 7f87dbfff700 1 -- 172.21.15.97:0/4123298616 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18435 (secure 0 0 0) 0x7f87e8099d20 con 0x7f87f41018b0 2023-10-08T09:53:53.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:53.235 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":77,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:53:17.122023+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:53:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 msgr2=0x7f87dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 0x7f87dc0a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87e4003730 tx=0x7f87e4000bc0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 msgr2=0x7f87f4196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87e800e9a0 tx=0x7f87e800ecb0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 shutdown_connections 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f87dc0a1480 0x7f87dc0a3940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f4102250 0x7f87f419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87f41006c0 0x7f87f4196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 --2- 172.21.15.97:0/4123298616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87f41018b0 0x7f87f4196b60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 >> 172.21.15.97:0/4123298616 conn(0x7f87f40fbc50 msgr2=0x7f87f4104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 shutdown_connections 2023-10-08T09:53:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.235+0000 7f87f90d5700 1 -- 172.21.15.97:0/4123298616 wait complete. 2023-10-08T09:53:53.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:53:53.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.387+0000 7f4440911700 1 -- 172.21.15.97:0/1154825828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ff6a0 msgr2=0x7f443c0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.387+0000 7f4440911700 1 --2- 172.21.15.97:0/1154825828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ff6a0 0x7f443c0ffac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f442c0099c0 tx=0x7f442c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 -- 172.21.15.97:0/1154825828 shutdown_connections 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 --2- 172.21.15.97:0/1154825828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 --2- 172.21.15.97:0/1154825828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ff6a0 0x7f443c0ffac0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 --2- 172.21.15.97:0/1154825828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c100890 0x7f443c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 -- 172.21.15.97:0/1154825828 >> 172.21.15.97:0/1154825828 conn(0x7f443c0fac10 msgr2=0x7f443c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 -- 172.21.15.97:0/1154825828 shutdown_connections 2023-10-08T09:53:53.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 -- 172.21.15.97:0/1154825828 wait complete. 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 Processor -- start 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.388+0000 7f4440911700 1 -- start start 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c0ff6a0 0x7f443c196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e270 con 0x7f443c100890 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e3b0 con 0x7f443c101230 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e540 con 0x7f443c0ff6a0 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55034/0 (socket says 172.21.15.97:55034) 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 -- 172.21.15.97:0/2195193953 learned_addr learned my addr 172.21.15.97:0/2195193953 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f443a59c700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c0ff6a0 0x7f443c196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 -- 172.21.15.97:0/2195193953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c0ff6a0 msgr2=0x7f443c196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c0ff6a0 0x7f443c196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f443ad9d700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 -- 172.21.15.97:0/2195193953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 msgr2=0x7f443c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 -- 172.21.15.97:0/2195193953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c0095f0 con 0x7f443c100890 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f443ad9d700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4439d9b700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f443000d950 tx=0x7f443000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430009940 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4430010460 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c19e820 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000f620 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.389+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c19ed30 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.390+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c106b40 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.391+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f44300108d0 con 0x7f443c100890 2023-10-08T09:53:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.392+0000 7f442b7fe700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 0x7f44240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.392+0000 7f443a59c700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 0x7f44240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.392+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44300cc6d0 con 0x7f443c100890 2023-10-08T09:53:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.392+0000 7f443a59c700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 0x7f44240a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442c003820 tx=0x7f442c005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.396+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f44300961d0 con 0x7f443c100890 2023-10-08T09:53:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:53 smithi178 ceph-mon[103156]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:53 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1797212049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:53 smithi186 ceph-mon[104393]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:53 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1797212049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:53:53.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.597+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f443c10c250 con 0x7f443c100890 2023-10-08T09:53:53.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.597+0000 7f442b7fe700 1 -- 172.21.15.97:0/2195193953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44300997f0 con 0x7f443c100890 2023-10-08T09:53:53.599 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:53:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 msgr2=0x7f44240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 0x7f44240a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442c003820 tx=0x7f442c005920 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 msgr2=0x7f443c196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f443000d950 tx=0x7f443000dc60 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 shutdown_connections 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44240a1480 0x7f44240a3940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c101230 0x7f443c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f443c0ff6a0 0x7f443c196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 --2- 172.21.15.97:0/2195193953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c100890 0x7f443c196c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 >> 172.21.15.97:0/2195193953 conn(0x7f443c0fac10 msgr2=0x7f443c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 shutdown_connections 2023-10-08T09:53:53.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.599+0000 7f4440911700 1 -- 172.21.15.97:0/2195193953 wait complete. 2023-10-08T09:53:53.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 -- 172.21.15.97:0/2509147352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc100790 msgr2=0x7fe1cc100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 --2- 172.21.15.97:0/2509147352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc100790 0x7fe1cc100bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0099c0 tx=0x7fe1bc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 -- 172.21.15.97:0/2509147352 shutdown_connections 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 --2- 172.21.15.97:0/2509147352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 0x7fe1cc1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 --2- 172.21.15.97:0/2509147352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 --2- 172.21.15.97:0/2509147352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc100790 0x7fe1cc100bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 -- 172.21.15.97:0/2509147352 >> 172.21.15.97:0/2509147352 conn(0x7fe1cc0fab10 msgr2=0x7fe1cc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 -- 172.21.15.97:0/2509147352 shutdown_connections 2023-10-08T09:53:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 -- 172.21.15.97:0/2509147352 wait complete. 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.736+0000 7fe1d217c700 1 Processor -- start 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 -- start start 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc100790 0x7fe1cc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 0x7fe1cc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19e0f0 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19e260 con 0x7fe1cc101130 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1d217c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc19e420 con 0x7fe1cc100790 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55046/0 (socket says 172.21.15.97:55046) 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 -- 172.21.15.97:0/3788805270 learned_addr learned my addr 172.21.15.97:0/3788805270 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1caffd700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc100790 0x7fe1cc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cbfff700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 0x7fe1cc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 -- 172.21.15.97:0/3788805270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc100790 msgr2=0x7fe1cc19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc100790 0x7fe1cc19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 -- 172.21.15.97:0/3788805270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 msgr2=0x7fe1cc19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 0x7fe1cc19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 -- 172.21.15.97:0/3788805270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0095f0 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.737+0000 7fe1cb7fe700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d980 tx=0x7fe1c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004020 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1c000f690 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc19e700 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0003a60 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc19ebe0 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.738+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc1069c0 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.739+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe1c000fb00 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.740+0000 7fe1c8ff9700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 0x7fe1b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.740+0000 7fe1caffd700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 0x7fe1b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.740+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1c00cc790 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.741+0000 7fe1caffd700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 0x7fe1b40a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009990 tx=0x7fe1bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:53.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.744+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1c0096290 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.956+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe1cc067450 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.956+0000 7fe1c8ff9700 1 -- 172.21.15.97:0/3788805270 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe1c00998b0 con 0x7fe1cc0ff5a0 2023-10-08T09:53:53.958 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:53:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-10-08T09:53:53.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T09:53:53.959 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-10-08T09:53:53.959 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-10-08T09:53:53.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:53:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-10-08T09:53:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 msgr2=0x7fe1b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 0x7fe1b40a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009990 tx=0x7fe1bc000bc0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 msgr2=0x7fe1cc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d980 tx=0x7fe1c000dc90 comp rx=0 tx=0).stop 2023-10-08T09:53:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 shutdown_connections 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1b40a1480 0x7fe1b40a3940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc101130 0x7fe1cc19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1cc0ff5a0 0x7fe1cc0683b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 --2- 172.21.15.97:0/3788805270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc100790 0x7fe1cc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.958+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 >> 172.21.15.97:0/3788805270 conn(0x7fe1cc0fab10 msgr2=0x7fe1cc103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.959+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 shutdown_connections 2023-10-08T09:53:53.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:53.959+0000 7fe1d217c700 1 -- 172.21.15.97:0/3788805270 wait complete. 2023-10-08T09:53:53.972 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T09:53:53.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T09:53:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 -- 172.21.15.97:0/3296251380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 msgr2=0x7fa63c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3296251380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c107ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b0d0 tx=0x7fa63400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 -- 172.21.15.97:0/3296251380 shutdown_connections 2023-10-08T09:53:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3296251380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c107ae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3296251380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 0x7fa63c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3296251380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 0x7fa63c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 -- 172.21.15.97:0/3296251380 >> 172.21.15.97:0/3296251380 conn(0x7fa63c0fac10 msgr2=0x7fa63c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 -- 172.21.15.97:0/3296251380 shutdown_connections 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.104+0000 7fa6444b2700 1 -- 172.21.15.97:0/3296251380 wait complete. 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 Processor -- start 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 -- start start 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 0x7fa63c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 0x7fa63c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19e340 con 0x7fa63c101230 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19e480 con 0x7fa63c100890 2023-10-08T09:53:54.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa6444b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19e610 con 0x7fa63c0ff6a0 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55064/0 (socket says 172.21.15.97:55064) 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 -- 172.21.15.97:0/3241652896 learned_addr learned my addr 172.21.15.97:0/3241652896 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa64224e700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 0x7fa63c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa641a4d700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 0x7fa63c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 -- 172.21.15.97:0/3241652896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 msgr2=0x7fa63c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 0x7fa63c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 -- 172.21.15.97:0/3241652896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 msgr2=0x7fa63c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 0x7fa63c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 -- 172.21.15.97:0/3241652896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634009d00 con 0x7fa63c101230 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.105+0000 7fa642a4f700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa634007b60 tx=0x7fa634008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.106+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400e040 con 0x7fa63c101230 2023-10-08T09:53:54.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.106+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63401d930 con 0x7fa63c101230 2023-10-08T09:53:54.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.106+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c19e890 con 0x7fa63c101230 2023-10-08T09:53:54.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.106+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63401ba00 con 0x7fa63c101230 2023-10-08T09:53:54.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.106+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c19ed10 con 0x7fa63c101230 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.107+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c106ad0 con 0x7fa63c101230 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.108+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa63401daa0 con 0x7fa63c101230 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.108+0000 7fa6337fe700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 0x7fa6280a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.109+0000 7fa64224e700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 0x7fa6280a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.109+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6340cdf30 con 0x7fa63c101230 2023-10-08T09:53:54.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.109+0000 7fa64224e700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 0x7fa6280a3a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa63c1006f0 tx=0x7fa62c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.112+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa634097b40 con 0x7fa63c101230 2023-10-08T09:53:54.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:54.308+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa63c10c060 con 0x7fa63c101230 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4123298616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2195193953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3788805270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:54 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4123298616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2195193953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3788805270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:54 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4123298616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2195193953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3788805270' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:54 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.177+0000 7fa6337fe700 1 -- 172.21.15.97:0/3241652896 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa63409b160 con 0x7fa63c101230 2023-10-08T09:53:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 msgr2=0x7fa6280a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 0x7fa6280a3a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa63c1006f0 tx=0x7fa62c009260 comp rx=0 tx=0).stop 2023-10-08T09:53:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 msgr2=0x7fa63c19dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa634007b60 tx=0x7fa634008ec0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 shutdown_connections 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa6280a15d0 0x7fa6280a3a90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c101230 0x7fa63c19dd70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa63c0ff6a0 0x7fa63c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 --2- 172.21.15.97:0/3241652896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa63c100890 0x7fa63c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 >> 172.21.15.97:0/3241652896 conn(0x7fa63c0fac10 msgr2=0x7fa63c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 shutdown_connections 2023-10-08T09:53:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.179+0000 7fa6444b2700 1 -- 172.21.15.97:0/3241652896 wait complete. 2023-10-08T09:53:55.183 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-10-08T09:53:55.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-08T09:53:55.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 -- 172.21.15.97:0/327348623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08101240 msgr2=0x7f8d08107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:55.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 --2- 172.21.15.97:0/327348623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08101240 0x7f8d08107af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000b0d0 tx=0x7f8d0000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 -- 172.21.15.97:0/327348623 shutdown_connections 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 --2- 172.21.15.97:0/327348623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d08101240 0x7f8d08107af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 --2- 172.21.15.97:0/327348623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d080ff6b0 0x7f8d080ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 --2- 172.21.15.97:0/327348623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d081008a0 0x7f8d08100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.328+0000 7f8d0e834700 1 -- 172.21.15.97:0/327348623 >> 172.21.15.97:0/327348623 conn(0x7f8d080fac00 msgr2=0x7f8d080fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- 172.21.15.97:0/327348623 shutdown_connections 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- 172.21.15.97:0/327348623 wait complete. 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 Processor -- start 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- start start 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:55.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d081008a0 0x7f8d08101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 0x7f8d081021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08102960 con 0x7f8d080ff6b0 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08102ad0 con 0x7f8d08101240 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d0e834700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d081a0890 con 0x7f8d081008a0 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.329+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55074/0 (socket says 172.21.15.97:55074) 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 -- 172.21.15.97:0/4142810088 learned_addr learned my addr 172.21.15.97:0/4142810088 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d077fe700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d081008a0 0x7f8d08101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d0cdd1700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 0x7f8d081021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 -- 172.21.15.97:0/4142810088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d081008a0 msgr2=0x7f8d08101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:55.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d081008a0 0x7f8d08101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 -- 172.21.15.97:0/4142810088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 msgr2=0x7f8d081021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 0x7f8d081021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 -- 172.21.15.97:0/4142810088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00009d00 con 0x7f8d080ff6b0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d0cdd1700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 0x7f8d081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d07fff700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00d980 tx=0x7f8cfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc004020 con 0x7f8d080ff6b0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d081a0b20 con 0x7f8d080ff6b0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cfc00f690 con 0x7f8d080ff6b0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc003a60 con 0x7f8d080ff6b0 2023-10-08T09:53:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.330+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d081a1080 con 0x7f8d080ff6b0 2023-10-08T09:53:55.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.331+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d08067980 con 0x7f8d080ff6b0 2023-10-08T09:53:55.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.332+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8cfc00fb00 con 0x7f8d080ff6b0 2023-10-08T09:53:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.333+0000 7f8d057fa700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 0x7f8cf00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.333+0000 7f8d077fe700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 0x7f8cf00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.333+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f8cfc0cc9a0 con 0x7f8d080ff6b0 2023-10-08T09:53:55.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.333+0000 7f8d077fe700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 0x7f8cf00a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005950 tx=0x7f8cf800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:55.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.336+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8cfc096290 con 0x7f8d080ff6b0 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:55 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:55.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:55.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:53:55.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:55.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:55.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:55.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:55 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:55 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:53:55.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:55.541+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f8d0810aed0 con 0x7f8d080ff6b0 2023-10-08T09:53:56.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.182+0000 7f8d057fa700 1 -- 172.21.15.97:0/4142810088 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7f8cfc0998b0 con 0x7f8d080ff6b0 2023-10-08T09:53:56.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.184+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 msgr2=0x7f8cf00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:56.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.184+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 0x7f8cf00a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005950 tx=0x7f8cf800b410 comp rx=0 tx=0).stop 2023-10-08T09:53:56.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.184+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 msgr2=0x7f8d081035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:56.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.184+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00d980 tx=0x7f8cfc00dc90 comp rx=0 tx=0).stop 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 shutdown_connections 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8cf00a1480 0x7f8cf00a3940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d08101240 0x7f8d081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d080ff6b0 0x7f8d081035f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 --2- 172.21.15.97:0/4142810088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d081008a0 0x7f8d08101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.189+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 >> 172.21.15.97:0/4142810088 conn(0x7f8d080fac00 msgr2=0x7f8d080fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.190+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 shutdown_connections 2023-10-08T09:53:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.190+0000 7f8d0e834700 1 -- 172.21.15.97:0/4142810088 wait complete. 2023-10-08T09:53:56.208 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-10-08T09:53:56.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T09:53:56.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 -- 172.21.15.97:0/4055695949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 msgr2=0x7f0e900ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:56.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/4055695949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e900ffaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e800099c0 tx=0x7f0e80009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 -- 172.21.15.97:0/4055695949 shutdown_connections 2023-10-08T09:53:56.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/4055695949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 0x7f0e90107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/4055695949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e900ffaf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/4055695949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 -- 172.21.15.97:0/4055695949 >> 172.21.15.97:0/4055695949 conn(0x7f0e900fac40 msgr2=0x7f0e900fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:56.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 -- 172.21.15.97:0/4055695949 shutdown_connections 2023-10-08T09:53:56.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.343+0000 7f0e95cf5700 1 -- 172.21.15.97:0/4055695949 wait complete. 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.344+0000 7f0e95cf5700 1 Processor -- start 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.344+0000 7f0e95cf5700 1 -- start start 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.344+0000 7f0e95cf5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e90195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 0x7f0e9019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d190 con 0x7f0e901008c0 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d2d0 con 0x7f0e900ff6d0 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d460 con 0x7f0e90101260 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55088/0 (socket says 172.21.15.97:55088) 2023-10-08T09:53:56.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 -- 172.21.15.97:0/2008232689 learned_addr learned my addr 172.21.15.97:0/2008232689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8f7fe700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e90195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8ffff700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 0x7f0e9019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 -- 172.21.15.97:0/2008232689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 msgr2=0x7f0e9019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 0x7f0e9019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 -- 172.21.15.97:0/2008232689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 msgr2=0x7f0e90195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e90195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 -- 172.21.15.97:0/2008232689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800095f0 con 0x7f0e901008c0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8effd700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9a0 tx=0x7f0e8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84009790 con 0x7f0e901008c0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9019d740 con 0x7f0e901008c0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e84010430 con 0x7f0e901008c0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84018680 con 0x7f0e901008c0 2023-10-08T09:53:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.345+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9019dc50 con 0x7f0e901008c0 2023-10-08T09:53:56.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.347+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90106bc0 con 0x7f0e901008c0 2023-10-08T09:53:56.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.348+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0e840105a0 con 0x7f0e901008c0 2023-10-08T09:53:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.348+0000 7f0e8cff9700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 0x7f0e780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.349+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f0e84014070 con 0x7f0e901008c0 2023-10-08T09:53:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.351+0000 7f0e8f7fe700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 0x7f0e780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.351+0000 7f0e8f7fe700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 0x7f0e780a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009fd0 tx=0x7f0e80019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.354+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e84052100 con 0x7f0e901008c0 2023-10-08T09:53:56.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:53:56.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:53:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:53:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:53:56.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:56 smithi097 ceph-mon[99491]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:53:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:56 smithi186 ceph-mon[104393]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:53:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:56.596+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0e901065a0 con 0x7f0e901008c0 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3241652896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4142810088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:53:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:56 smithi178 ceph-mon[103156]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:53:57.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.197+0000 7f0e8cff9700 1 -- 172.21.15.97:0/2008232689 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0e84099fa0 con 0x7f0e901008c0 2023-10-08T09:53:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 msgr2=0x7f0e780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 0x7f0e780a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009fd0 tx=0x7f0e80019040 comp rx=0 tx=0).stop 2023-10-08T09:53:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 msgr2=0x7f0e90195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9a0 tx=0x7f0e8400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 shutdown_connections 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0e780a1480 0x7f0e780a3940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e90101260 0x7f0e9019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e900ff6d0 0x7f0e90195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 --2- 172.21.15.97:0/2008232689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e901008c0 0x7f0e90195ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.200+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 >> 172.21.15.97:0/2008232689 conn(0x7f0e900fac40 msgr2=0x7f0e90103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.201+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 shutdown_connections 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.201+0000 7f0e95cf5700 1 -- 172.21.15.97:0/2008232689 wait complete. 2023-10-08T09:53:57.203 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T09:53:57.220 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-08T09:53:57.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-08T09:53:57.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 -- 172.21.15.97:0/3466159201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d94101230 msgr2=0x7f2d94107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/3466159201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d94101230 0x7f2d94107ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00b0d0 tx=0x7f2d8c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 -- 172.21.15.97:0/3466159201 shutdown_connections 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/3466159201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d94101230 0x7f2d94107ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/3466159201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d940ff6a0 0x7f2d940ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/3466159201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 0x7f2d94100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 -- 172.21.15.97:0/3466159201 >> 172.21.15.97:0/3466159201 conn(0x7f2d940fac10 msgr2=0x7f2d940fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.350+0000 7f2d98d0b700 1 -- 172.21.15.97:0/3466159201 shutdown_connections 2023-10-08T09:53:57.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 -- 172.21.15.97:0/3466159201 wait complete. 2023-10-08T09:53:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 Processor -- start 2023-10-08T09:53:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 -- start start 2023-10-08T09:53:57.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 0x7f2d9419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d94101230 0x7f2d9419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941a0510 con 0x7f2d940ff6a0 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941a0680 con 0x7f2d94101230 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.351+0000 7f2d98d0b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941a0840 con 0x7f2d94100890 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55106/0 (socket says 172.21.15.97:55106) 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 -- 172.21.15.97:0/1634661942 learned_addr learned my addr 172.21.15.97:0/1634661942 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d91d9b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 0x7f2d9419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d92d9d700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d94101230 0x7f2d9419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:57.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 -- 172.21.15.97:0/1634661942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 msgr2=0x7f2d9419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 0x7f2d9419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 -- 172.21.15.97:0/1634661942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d94101230 msgr2=0x7f2d9419feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d94101230 0x7f2d9419feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 -- 172.21.15.97:0/1634661942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c009d00 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d9259c700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d980 tx=0x7f2d8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400b950 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d84007620 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d941a0b20 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400f3c0 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.352+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941a1030 con 0x7f2d940ff6a0 2023-10-08T09:53:57.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.353+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d94106a50 con 0x7f2d940ff6a0 2023-10-08T09:53:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.355+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2d84010640 con 0x7f2d940ff6a0 2023-10-08T09:53:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.355+0000 7f2d837fe700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 0x7f2d7c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.355+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d840cd330 con 0x7f2d940ff6a0 2023-10-08T09:53:57.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.355+0000 7f2d91d9b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 0x7f2d7c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:57.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.356+0000 7f2d91d9b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 0x7f2d7c0a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d94068910 tx=0x7f2d8800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:57.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.359+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d84096ae0 con 0x7f2d940ff6a0 2023-10-08T09:53:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:57 smithi178 ceph-mon[103156]: pgmap v253: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:53:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:57 smithi178 ceph-mon[103156]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:53:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:57 smithi178 ceph-mon[103156]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:53:57.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:57 smithi178 ceph-mon[103156]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:53:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:57 smithi186 ceph-mon[104393]: pgmap v253: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:53:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:57 smithi186 ceph-mon[104393]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:53:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:57 smithi186 ceph-mon[104393]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:53:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:57 smithi186 ceph-mon[104393]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:53:57.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.577+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2d94067500 con 0x7f2d940ff6a0 2023-10-08T09:53:57.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:57 smithi097 ceph-mon[99491]: pgmap v253: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:53:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:57 smithi097 ceph-mon[99491]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:53:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:57 smithi097 ceph-mon[99491]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:53:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:57 smithi097 ceph-mon[99491]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:53:57.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.877+0000 7f2d837fe700 1 -- 172.21.15.97:0/1634661942 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2d8409a100 con 0x7f2d940ff6a0 2023-10-08T09:53:57.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 msgr2=0x7f2d7c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 0x7f2d7c0a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d94068910 tx=0x7f2d8800b3c0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 msgr2=0x7f2d94068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d980 tx=0x7f2d8400dc90 comp rx=0 tx=0).stop 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 shutdown_connections 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d7c0a1480 0x7f2d7c0a3940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d94101230 0x7f2d9419feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d940ff6a0 0x7f2d94068350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 --2- 172.21.15.97:0/1634661942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d94100890 0x7f2d9419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 >> 172.21.15.97:0/1634661942 conn(0x7f2d940fac10 msgr2=0x7f2d94103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 shutdown_connections 2023-10-08T09:53:57.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:57.879+0000 7f2d98d0b700 1 -- 172.21.15.97:0/1634661942 wait complete. 2023-10-08T09:53:57.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T09:53:58.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 -- 172.21.15.97:0/637256357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 msgr2=0x7f98000ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:58.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 --2- 172.21.15.97:0/637256357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f98000ff9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97f00099c0 tx=0x7f97f0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 -- 172.21.15.97:0/637256357 shutdown_connections 2023-10-08T09:53:58.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 --2- 172.21.15.97:0/637256357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9800101140 0x7f98001079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 --2- 172.21.15.97:0/637256357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f98000ff9d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 --2- 172.21.15.97:0/637256357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98001007a0 0x7f9800100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 -- 172.21.15.97:0/637256357 >> 172.21.15.97:0/637256357 conn(0x7f98000fab20 msgr2=0x7f98000fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 -- 172.21.15.97:0/637256357 shutdown_connections 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.023+0000 7f980768a700 1 -- 172.21.15.97:0/637256357 wait complete. 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 Processor -- start 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 -- start start 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98001007a0 0x7f980019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:58.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9800101140 0x7f980019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019e240 con 0x7f98000ff5b0 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019e3b0 con 0x7f98001007a0 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f980768a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019e570 con 0x7f9800101140 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55120/0 (socket says 172.21.15.97:55120) 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 -- 172.21.15.97:0/849898656 learned_addr learned my addr 172.21.15.97:0/849898656 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9804c25700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98001007a0 0x7f980019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805c27700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9800101140 0x7f980019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:58.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 -- 172.21.15.97:0/849898656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9800101140 msgr2=0x7f980019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9800101140 0x7f980019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 -- 172.21.15.97:0/849898656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98001007a0 msgr2=0x7f980019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98001007a0 0x7f980019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.024+0000 7f9805426700 1 -- 172.21.15.97:0/849898656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00095f0 con 0x7f98000ff5b0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f9805426700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009f50 tx=0x7f97f0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f001d070 con 0x7f98000ff5b0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97f0022470 con 0x7f98000ff5b0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980019e790 con 0x7f98000ff5b0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0022ea0 con 0x7f98000ff5b0 2023-10-08T09:53:58.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.025+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980019ecc0 con 0x7f98000ff5b0 2023-10-08T09:53:58.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.026+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98001069d0 con 0x7f98000ff5b0 2023-10-08T09:53:58.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.027+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f97f0022800 con 0x7f98000ff5b0 2023-10-08T09:53:58.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.027+0000 7f97f67fc700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 0x7f97ec0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:53:58.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.027+0000 7f9804c25700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 0x7f97ec0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:53:58.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.027+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f97f00ce1b0 con 0x7f98000ff5b0 2023-10-08T09:53:58.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.028+0000 7f9804c25700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 0x7f97ec0a3930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006fd0 tx=0x7f97fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:53:58.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.031+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f97f00978c0 con 0x7f98000ff5b0 2023-10-08T09:53:58.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:53:58.226+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9800067450 con 0x7f98000ff5b0 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:53:58.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:58 smithi186 ceph-mon[104393]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:58.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:58.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:53:58.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:53:58.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:58 smithi097 ceph-mon[99491]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2008232689' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1634661942' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:53:58.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:58 smithi178 ceph-mon[103156]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: pgmap v256: 97 pgs: 18 creating+peering, 51 unknown, 28 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:53:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:53:59 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T09:53:59.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: pgmap v256: 97 pgs: 18 creating+peering, 51 unknown, 28 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:53:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:53:59 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: pgmap v256: 97 pgs: 18 creating+peering, 51 unknown, 28 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:53:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:53:59 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T09:54:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.222+0000 7f97f67fc700 1 -- 172.21.15.97:0/849898656 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) v1 ==== 228+0+0 (secure 0 0 0) 0x7f97f009aee0 con 0x7f98000ff5b0 2023-10-08T09:54:00.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.227+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 msgr2=0x7f97ec0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:00.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.227+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 0x7f97ec0a3930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006fd0 tx=0x7f97fc008040 comp rx=0 tx=0).stop 2023-10-08T09:54:00.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.227+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 msgr2=0x7f9800068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:00.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.227+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009f50 tx=0x7f97f0003970 comp rx=0 tx=0).stop 2023-10-08T09:54:00.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 shutdown_connections 2023-10-08T09:54:00.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f97ec0a1470 0x7f97ec0a3930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9800101140 0x7f980019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98000ff5b0 0x7f9800068370 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 --2- 172.21.15.97:0/849898656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98001007a0 0x7f980019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.231+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 >> 172.21.15.97:0/849898656 conn(0x7f98000fab20 msgr2=0x7f9800103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:00.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.232+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 shutdown_connections 2023-10-08T09:54:00.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.233+0000 7f980768a700 1 -- 172.21.15.97:0/849898656 wait complete. 2023-10-08T09:54:00.235 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' created 2023-10-08T09:54:00.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 -- 172.21.15.97:0/1617888901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 msgr2=0x7f437c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 --2- 172.21.15.97:0/1617888901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c101170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4370009990 tx=0x7f4370009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 -- 172.21.15.97:0/1617888901 shutdown_connections 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 --2- 172.21.15.97:0/1617888901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 --2- 172.21.15.97:0/1617888901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c101170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 --2- 172.21.15.97:0/1617888901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 0x7f437c0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 -- 172.21.15.97:0/1617888901 >> 172.21.15.97:0/1617888901 conn(0x7f437c0fa3c0 msgr2=0x7f437c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:00.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 -- 172.21.15.97:0/1617888901 shutdown_connections 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.379+0000 7f4380870700 1 -- 172.21.15.97:0/1617888901 wait complete. 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 Processor -- start 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 -- start start 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 0x7f437c1955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19d0f0 con 0x7f437c100d50 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19d230 con 0x7f437c0ff910 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4380870700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19d3c0 con 0x7f437c1002d0 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:00.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55144/0 (socket says 172.21.15.97:55144) 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 -- 172.21.15.97:0/1679850368 learned_addr learned my addr 172.21.15.97:0/1679850368 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437a59c700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 0x7f437c1955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4379d9b700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 -- 172.21.15.97:0/1679850368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 msgr2=0x7f437c195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 -- 172.21.15.97:0/1679850368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 msgr2=0x7f437c1955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 0x7f437c1955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f437ad9d700 1 -- 172.21.15.97:0/1679850368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700095f0 con 0x7f437c100d50 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.380+0000 7f4379d9b700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f437ad9d700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4374007ee0 tx=0x7f437400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374017070 con 0x7f437c100d50 2023-10-08T09:54:00.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437c19d6a0 con 0x7f437c100d50 2023-10-08T09:54:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f437400f040 con 0x7f437c100d50 2023-10-08T09:54:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374005740 con 0x7f437c100d50 2023-10-08T09:54:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.381+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437c19dbb0 con 0x7f437c100d50 2023-10-08T09:54:00.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.382+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437c106ae0 con 0x7f437c100d50 2023-10-08T09:54:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.383+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4374022020 con 0x7f437c100d50 2023-10-08T09:54:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.384+0000 7f436b7fe700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 0x7f43640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.384+0000 7f437a59c700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 0x7f43640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.384+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f43740cc590 con 0x7f437c100d50 2023-10-08T09:54:00.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.384+0000 7f437a59c700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 0x7f43640a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f437000b5c0 tx=0x7f4370003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:00.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.388+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4374095a20 con 0x7f437c100d50 2023-10-08T09:54:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:00 smithi186 ceph-mon[104393]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:54:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:00.578+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f437c067980 con 0x7f437c100d50 2023-10-08T09:54:00.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:00 smithi097 ceph-mon[99491]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:54:00.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:00 smithi178 ceph-mon[103156]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:54:01.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.224+0000 7f436b7fe700 1 -- 172.21.15.97:0/1679850368 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4374099040 con 0x7f437c100d50 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 msgr2=0x7f43640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 0x7f43640a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f437000b5c0 tx=0x7f4370003790 comp rx=0 tx=0).stop 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 msgr2=0x7f437c19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4374007ee0 tx=0x7f437400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 shutdown_connections 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f43640a1480 0x7f43640a3940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f437c100d50 0x7f437c19cb20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f437c0ff910 0x7f437c1955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 --2- 172.21.15.97:0/1679850368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f437c1002d0 0x7f437c195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 >> 172.21.15.97:0/1679850368 conn(0x7f437c0fa3c0 msgr2=0x7f437c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 shutdown_connections 2023-10-08T09:54:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.227+0000 7f4380870700 1 -- 172.21.15.97:0/1679850368 wait complete. 2023-10-08T09:54:01.229 INFO:teuthology.orchestra.run.smithi097.stderr:set pool 4 allow_ec_overwrites to true 2023-10-08T09:54:01.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-08T09:54:01.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 -- 172.21.15.97:0/1076346705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fff70 msgr2=0x7fb3401003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 --2- 172.21.15.97:0/1076346705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fff70 0x7fb3401003f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb334009990 tx=0x7fb334009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 -- 172.21.15.97:0/1076346705 shutdown_connections 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 --2- 172.21.15.97:0/1076346705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb340100930 0x7fb340107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 --2- 172.21.15.97:0/1076346705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fee20 0x7fb3400ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 --2- 172.21.15.97:0/1076346705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fff70 0x7fb3401003f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 -- 172.21.15.97:0/1076346705 >> 172.21.15.97:0/1076346705 conn(0x7fb3400fa390 msgr2=0x7fb3400fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 -- 172.21.15.97:0/1076346705 shutdown_connections 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.376+0000 7fb346039700 1 -- 172.21.15.97:0/1076346705 wait complete. 2023-10-08T09:54:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.377+0000 7fb346039700 1 Processor -- start 2023-10-08T09:54:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.377+0000 7fb346039700 1 -- start start 2023-10-08T09:54:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fff70 0x7fb34019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb340100930 0x7fb34019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e1e0 con 0x7fb3400fee20 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e350 con 0x7fb3400fff70 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e510 con 0x7fb340100930 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55158/0 (socket says 172.21.15.97:55158) 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 -- 172.21.15.97:0/1150729385 learned_addr learned my addr 172.21.15.97:0/1150729385 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33effd700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fff70 0x7fb34019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 -- 172.21.15.97:0/1150729385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb340100930 msgr2=0x7fb34019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb340100930 0x7fb34019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 -- 172.21.15.97:0/1150729385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fff70 msgr2=0x7fb34019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fff70 0x7fb34019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 -- 172.21.15.97:0/1150729385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33f7fe700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c9b0 tx=0x7fb33000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330007b00 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb330007c60 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34019e790 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330018660 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.378+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34019eca0 con 0x7fb3400fee20 2023-10-08T09:54:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.379+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3401061c0 con 0x7fb3400fee20 2023-10-08T09:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.380+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb330018880 con 0x7fb3400fee20 2023-10-08T09:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.381+0000 7fb33cff9700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 0x7fb3280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.381+0000 7fb33effd700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 0x7fb3280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.381+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb3300ccf40 con 0x7fb3400fee20 2023-10-08T09:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.382+0000 7fb33effd700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 0x7fb3280a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3340037f0 tx=0x7fb334005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.386+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb3300963d0 con 0x7fb3400fee20 2023-10-08T09:54:01.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:54:01 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: 2023-10-08T09:54:01.162+0000 7f793dc9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T09:54:01.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:01 smithi178 ceph-mon[103156]: pgmap v259: 97 pgs: 18 creating+peering, 34 unknown, 45 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-10-08T09:54:01.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:54:01.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:01 smithi178 ceph-mon[103156]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:54:01.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:54:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:01 smithi186 ceph-mon[104393]: pgmap v259: 97 pgs: 18 creating+peering, 34 unknown, 45 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-10-08T09:54:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:54:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:01 smithi186 ceph-mon[104393]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:54:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:54:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:01.621+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb340066ca0 con 0x7fb3400fee20 2023-10-08T09:54:01.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:01 smithi097 ceph-mon[99491]: pgmap v259: 97 pgs: 18 creating+peering, 34 unknown, 45 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2023-10-08T09:54:01.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/849898656' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:54:01.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:01 smithi097 ceph-mon[99491]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:54:01.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:54:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.244+0000 7fb33cff9700 1 -- 172.21.15.97:0/1150729385 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb3300999f0 con 0x7fb3400fee20 2023-10-08T09:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 msgr2=0x7fb3280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 0x7fb3280a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3340037f0 tx=0x7fb334005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 msgr2=0x7fb3401086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c9b0 tx=0x7fb33000cd70 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 shutdown_connections 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb3280a1480 0x7fb3280a3940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb340100930 0x7fb34019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3400fee20 0x7fb3401086c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 --2- 172.21.15.97:0/1150729385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3400fff70 0x7fb34019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.246+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 >> 172.21.15.97:0/1150729385 conn(0x7fb3400fa390 msgr2=0x7fb340103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.247+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 shutdown_connections 2023-10-08T09:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.247+0000 7fb346039700 1 -- 172.21.15.97:0/1150729385 wait complete. 2023-10-08T09:54:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:added data pool 4 to fsmap 2023-10-08T09:54:02.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.384+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2080234323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c4100790 msgr2=0x7f23c4100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.384+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2080234323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c4100790 0x7f23c4100bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23b80099c0 tx=0x7f23b8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2080234323 shutdown_connections 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2080234323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4101130 0x7f23c41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2080234323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c40ff5a0 0x7f23c40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2080234323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c4100790 0x7f23c4100bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2080234323 >> 172.21.15.97:0/2080234323 conn(0x7f23c40fab10 msgr2=0x7f23c40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:02.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2080234323 shutdown_connections 2023-10-08T09:54:02.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2080234323 wait complete. 2023-10-08T09:54:02.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 Processor -- start 2023-10-08T09:54:02.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 -- start start 2023-10-08T09:54:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.385+0000 7f23ca2ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 0x7f23c419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c4101130 0x7f23c419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e0a0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e210 con 0x7f23c4101130 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e3d0 con 0x7f23c4100790 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55172/0 (socket says 172.21.15.97:55172) 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 -- 172.21.15.97:0/2776926806 learned_addr learned my addr 172.21.15.97:0/2776926806 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c37fe700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 0x7f23c419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 -- 172.21.15.97:0/2776926806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 msgr2=0x7f23c419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 0x7f23c419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 -- 172.21.15.97:0/2776926806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c4101130 msgr2=0x7f23c419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c889c700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c4101130 0x7f23c419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c4101130 0x7f23c419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 -- 172.21.15.97:0/2776926806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80095f0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c37fe700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 0x7f23c419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c3fff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c970 tx=0x7f23b400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4004220 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b4007d60 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c419e6b0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4003d00 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.386+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c419ebc0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.387+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c41069d0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.388+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f23b4004380 con 0x7f23c40ff5a0 2023-10-08T09:54:02.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.389+0000 7f23c17fa700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 0x7f23ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.389+0000 7f23c37fe700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 0x7f23ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.389+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f23b40cceb0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.389+0000 7f23c37fe700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 0x7f23ac0a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006010 tx=0x7f23b8009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:02.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.392+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f23b40962c0 con 0x7f23c40ff5a0 2023-10-08T09:54:02.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:54:02.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:02 smithi186 ceph-mon[104393]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:54:02.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:54:02.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.633+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23c4067450 con 0x7f23c40ff5a0 2023-10-08T09:54:02.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.634+0000 7f23c17fa700 1 -- 172.21.15.97:0/2776926806 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f23b401d090 con 0x7f23c40ff5a0 2023-10-08T09:54:02.635 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:02.636 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:01.623120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T09:54:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 msgr2=0x7f23ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 0x7f23ac0a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006010 tx=0x7f23b8009530 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 msgr2=0x7f23c40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c970 tx=0x7f23b400cd30 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 shutdown_connections 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23ac0a1480 0x7f23ac0a3940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23c4101130 0x7f23c419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c40ff5a0 0x7f23c40683b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 --2- 172.21.15.97:0/2776926806 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23c4100790 0x7f23c419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.636+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 >> 172.21.15.97:0/2776926806 conn(0x7f23c40fab10 msgr2=0x7f23c4103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.637+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 shutdown_connections 2023-10-08T09:54:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.637+0000 7f23ca2ff700 1 -- 172.21.15.97:0/2776926806 wait complete. 2023-10-08T09:54:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2023-10-08T09:54:02.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:54:02.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:02 smithi097 ceph-mon[99491]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:54:02.648 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:54:02.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:02.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1679850368' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:54:02.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:02 smithi178 ceph-mon[103156]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:54:02.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:54:02.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 -- 172.21.15.97:0/1183579576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350101190 msgr2=0x7f2350103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 --2- 172.21.15.97:0/1183579576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350101190 0x7f2350103580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2340009990 tx=0x7f2340009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 -- 172.21.15.97:0/1183579576 shutdown_connections 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 --2- 172.21.15.97:0/1183579576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23501063f0 0x7f2350108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 --2- 172.21.15.97:0/1183579576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350101190 0x7f2350103580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 --2- 172.21.15.97:0/1183579576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2350103ac0 0x7f2350105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 -- 172.21.15.97:0/1183579576 >> 172.21.15.97:0/1183579576 conn(0x7f23500fab40 msgr2=0x7f23500fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.769+0000 7f2354d75700 1 -- 172.21.15.97:0/1183579576 shutdown_connections 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 -- 172.21.15.97:0/1183579576 wait complete. 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 Processor -- start 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 -- start start 2023-10-08T09:54:02.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2350101190 0x7f23501965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23501063f0 0x7f235019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235019e120 con 0x7f2350103ac0 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235019e260 con 0x7f23501063f0 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f2354d75700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235019e3f0 con 0x7f2350101190 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.770+0000 7f234dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55188/0 (socket says 172.21.15.97:55188) 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 -- 172.21.15.97:0/715775381 learned_addr learned my addr 172.21.15.97:0/715775381 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234e59c700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2350101190 0x7f23501965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234ed9d700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23501063f0 0x7f235019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 -- 172.21.15.97:0/715775381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2350101190 msgr2=0x7f23501965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2350101190 0x7f23501965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 -- 172.21.15.97:0/715775381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23501063f0 msgr2=0x7f235019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23501063f0 0x7f235019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 -- 172.21.15.97:0/715775381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400095f0 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f234dd9b700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f234400d950 tx=0x7f234400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344009b40 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235019e6d0 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2344009ca0 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234400f620 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.771+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235019ec30 con 0x7f2350103ac0 2023-10-08T09:54:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.772+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500fc6c0 con 0x7f2350103ac0 2023-10-08T09:54:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.773+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2344010460 con 0x7f2350103ac0 2023-10-08T09:54:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.773+0000 7f233f7fe700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 0x7f23380a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.774+0000 7f234e59c700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 0x7f23380a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.774+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f23440cd2b0 con 0x7f2350103ac0 2023-10-08T09:54:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.774+0000 7f234e59c700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 0x7f23380a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f234000b5c0 tx=0x7f2340005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:02.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.777+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2344096720 con 0x7f2350103ac0 2023-10-08T09:54:02.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.970+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f235010bed0 con 0x7f2350103ac0 2023-10-08T09:54:02.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.971+0000 7f233f7fe700 1 -- 172.21.15.97:0/715775381 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f2344016090 con 0x7f2350103ac0 2023-10-08T09:54:02.972 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:02.972 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 msgr2=0x7f23380a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 0x7f23380a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f234000b5c0 tx=0x7f2340005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 msgr2=0x7f2350196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f234400d950 tx=0x7f234400dd10 comp rx=0 tx=0).stop 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 shutdown_connections 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f23380a1480 0x7f23380a3940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23501063f0 0x7f235019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2350101190 0x7f23501965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 --2- 172.21.15.97:0/715775381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2350103ac0 0x7f2350196b30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:02.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 >> 172.21.15.97:0/715775381 conn(0x7f23500fab40 msgr2=0x7f2350101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:02.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 shutdown_connections 2023-10-08T09:54:02.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:02.974+0000 7f2354d75700 1 -- 172.21.15.97:0/715775381 wait complete. 2023-10-08T09:54:02.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 -- 172.21.15.97:0/3656828021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 msgr2=0x7fc364100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/3656828021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc364100c00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3580099c0 tx=0x7fc358009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 -- 172.21.15.97:0/3656828021 shutdown_connections 2023-10-08T09:54:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/3656828021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 0x7fc3641079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/3656828021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/3656828021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc364100c00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 -- 172.21.15.97:0/3656828021 >> 172.21.15.97:0/3656828021 conn(0x7fc3640fab40 msgr2=0x7fc3640fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 -- 172.21.15.97:0/3656828021 shutdown_connections 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.107+0000 7fc36a9c4700 1 -- 172.21.15.97:0/3656828021 wait complete. 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 Processor -- start 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 -- start start 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 0x7fc36419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36419e150 con 0x7fc3641007a0 2023-10-08T09:54:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36419e2c0 con 0x7fc3640ff5b0 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc36a9c4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36419e480 con 0x7fc364101140 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55202/0 (socket says 172.21.15.97:55202) 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 -- 172.21.15.97:0/2044977536 learned_addr learned my addr 172.21.15.97:0/2044977536 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc363fff700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc368f61700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 0x7fc36419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 -- 172.21.15.97:0/2044977536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 msgr2=0x7fc36419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 0x7fc36419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 -- 172.21.15.97:0/2044977536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 msgr2=0x7fc3640695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc3637fe700 1 -- 172.21.15.97:0/2044977536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580095f0 con 0x7fc3641007a0 2023-10-08T09:54:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.108+0000 7fc363fff700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc3637fe700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc358009fd0 tx=0x7fc358005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35801d070 con 0x7fc3641007a0 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3580059a0 con 0x7fc3641007a0 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36419e6a0 con 0x7fc3641007a0 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358017650 con 0x7fc3641007a0 2023-10-08T09:54:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.109+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36419eb50 con 0x7fc3641007a0 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.110+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3641069d0 con 0x7fc3641007a0 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.111+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc358022470 con 0x7fc3641007a0 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.112+0000 7fc3617fa700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 0x7fc34c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.112+0000 7fc363fff700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 0x7fc34c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.112+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc3580ce960 con 0x7fc3641007a0 2023-10-08T09:54:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.112+0000 7fc363fff700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 0x7fc34c0a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc354005950 tx=0x7fc3540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.116+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc358097f80 con 0x7fc3641007a0 2023-10-08T09:54:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.308+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc364066bc0 con 0x7fc3641007a0 2023-10-08T09:54:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.310+0000 7fc3617fa700 1 -- 172.21.15.97:0/2044977536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7fc35800fe60 con 0x7fc3641007a0 2023-10-08T09:54:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:03.311 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 msgr2=0x7fc34c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 0x7fc34c0a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc354005950 tx=0x7fc3540058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 msgr2=0x7fc36419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc358009fd0 tx=0x7fc358005f10 comp rx=0 tx=0).stop 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 shutdown_connections 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc34c0a1480 0x7fc34c0a3940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc364101140 0x7fc36419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3640ff5b0 0x7fc3640695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 --2- 172.21.15.97:0/2044977536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3641007a0 0x7fc36419b340 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 >> 172.21.15.97:0/2044977536 conn(0x7fc3640fab40 msgr2=0x7fc364103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 shutdown_connections 2023-10-08T09:54:03.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.312+0000 7fc36a9c4700 1 -- 172.21.15.97:0/2044977536 wait complete. 2023-10-08T09:54:03.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:54:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.460+0000 7f18d2e09700 1 -- 172.21.15.97:0/2453323520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 msgr2=0x7f18cc0ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.460+0000 7f18d2e09700 1 --2- 172.21.15.97:0/2453323520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc0ff9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18c00099c0 tx=0x7f18c0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- 172.21.15.97:0/2453323520 shutdown_connections 2023-10-08T09:54:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- 172.21.15.97:0/2453323520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 0x7f18cc1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- 172.21.15.97:0/2453323520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc0ff9d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- 172.21.15.97:0/2453323520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- 172.21.15.97:0/2453323520 >> 172.21.15.97:0/2453323520 conn(0x7f18cc0fab40 msgr2=0x7f18cc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- 172.21.15.97:0/2453323520 shutdown_connections 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- 172.21.15.97:0/2453323520 wait complete. 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 Processor -- start 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- start start 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 0x7f18cc19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc19e110 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc19e280 con 0x7f18cc1007a0 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d2e09700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc19e440 con 0x7f18cc101140 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d0ba5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d0ba5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55214/0 (socket says 172.21.15.97:55214) 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.461+0000 7f18d0ba5700 1 -- 172.21.15.97:0/509361922 learned_addr learned my addr 172.21.15.97:0/509361922 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d13a6700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 0x7f18cc19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18cbfff700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 -- 172.21.15.97:0/509361922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 msgr2=0x7f18cc19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 0x7f18cc19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 -- 172.21.15.97:0/509361922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 msgr2=0x7f18cc19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 -- 172.21.15.97:0/509361922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00095f0 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18cbfff700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d0ba5700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f18c0003820 tx=0x7f18c0005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c001d070 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18c0004110 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc19e6c0 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0022e40 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.462+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc19eb70 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.463+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc106a80 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.464+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f18c0022570 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.465+0000 7f18c9ffb700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 0x7f18bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.465+0000 7f18cbfff700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 0x7f18bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.465+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f18c00ce1e0 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.465+0000 7f18cbfff700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 0x7f18bc0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005fd0 tx=0x7f18b8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.469+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f18c0097780 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: pgmap v262: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2776926806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:03 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/715775381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: pgmap v262: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2776926806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/715775381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: pgmap v262: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1150729385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2776926806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:03 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/715775381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.711+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18cc066bc0 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.711+0000 7f18c9ffb700 1 -- 172.21.15.97:0/509361922 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f18c009ada0 con 0x7f18cc0ff5b0 2023-10-08T09:54:03.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:03.713 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24536,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:01.623120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T09:54:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.713+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 msgr2=0x7f18bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.713+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 0x7f18bc0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005fd0 tx=0x7f18b8005dc0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.713+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 msgr2=0x7f18cc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f18c0003820 tx=0x7f18c0005ad0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 shutdown_connections 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f18bc0a1480 0x7f18bc0a3940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18cc101140 0x7f18cc19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18cc0ff5b0 0x7f18cc069560 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 --2- 172.21.15.97:0/509361922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18cc1007a0 0x7f18cc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 >> 172.21.15.97:0/509361922 conn(0x7f18cc0fab40 msgr2=0x7f18cc103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 shutdown_connections 2023-10-08T09:54:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.714+0000 7f18d2e09700 1 -- 172.21.15.97:0/509361922 wait complete. 2023-10-08T09:54:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2023-10-08T09:54:03.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- 172.21.15.97:0/891126990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c101130 msgr2=0x7f636c1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 --2- 172.21.15.97:0/891126990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c101130 0x7f636c1079e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f636400b0d0 tx=0x7f636400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- 172.21.15.97:0/891126990 shutdown_connections 2023-10-08T09:54:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 --2- 172.21.15.97:0/891126990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c101130 0x7f636c1079e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 --2- 172.21.15.97:0/891126990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c0ff5a0 0x7f636c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 --2- 172.21.15.97:0/891126990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f636c100790 0x7f636c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- 172.21.15.97:0/891126990 >> 172.21.15.97:0/891126990 conn(0x7f636c0fab10 msgr2=0x7f636c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- 172.21.15.97:0/891126990 shutdown_connections 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- 172.21.15.97:0/891126990 wait complete. 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 Processor -- start 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 -- start start 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.847+0000 7f63743ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f636c100790 0x7f636c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c101130 0x7f636c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19e230 con 0x7f636c0ff5a0 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19e370 con 0x7f636c100790 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19e500 con 0x7f636c101130 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55228/0 (socket says 172.21.15.97:55228) 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 -- 172.21.15.97:0/1899297920 learned_addr learned my addr 172.21.15.97:0/1899297920 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 -- 172.21.15.97:0/1899297920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c101130 msgr2=0x7f636c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c101130 0x7f636c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 -- 172.21.15.97:0/1899297920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f636c100790 msgr2=0x7f636c196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f636c100790 0x7f636c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 -- 172.21.15.97:0/1899297920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364009d00 con 0x7f636c0ff5a0 2023-10-08T09:54:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f637218a700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c970 tx=0x7f635c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c004220 con 0x7f636c0ff5a0 2023-10-08T09:54:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c19e7e0 con 0x7f636c0ff5a0 2023-10-08T09:54:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c007d60 con 0x7f636c0ff5a0 2023-10-08T09:54:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c003d00 con 0x7f636c0ff5a0 2023-10-08T09:54:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.848+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c19ed40 con 0x7f636c0ff5a0 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.849+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c067450 con 0x7f636c0ff5a0 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.850+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f635c004380 con 0x7f636c0ff5a0 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.851+0000 7f63637fe700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 0x7f63580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.851+0000 7f6371989700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 0x7f63580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.851+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f635c0cced0 con 0x7f636c0ff5a0 2023-10-08T09:54:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.851+0000 7f6371989700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 0x7f63580a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6368005950 tx=0x7f63680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:03.854+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f635c0962e0 con 0x7f636c0ff5a0 2023-10-08T09:54:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.047+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f636c10b070 con 0x7f636c0ff5a0 2023-10-08T09:54:04.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.048+0000 7f63637fe700 1 -- 172.21.15.97:0/1899297920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f635c01d090 con 0x7f636c0ff5a0 2023-10-08T09:54:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:04.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 msgr2=0x7f63580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 0x7f63580a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6368005950 tx=0x7f63680058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 msgr2=0x7f636c196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c970 tx=0x7f635c00cd30 comp rx=0 tx=0).stop 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 shutdown_connections 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f63580a1480 0x7f63580a3940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f636c101130 0x7f636c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636c0ff5a0 0x7f636c196700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 --2- 172.21.15.97:0/1899297920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f636c100790 0x7f636c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 >> 172.21.15.97:0/1899297920 conn(0x7f636c0fab10 msgr2=0x7f636c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 shutdown_connections 2023-10-08T09:54:04.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.051+0000 7f63743ee700 1 -- 172.21.15.97:0/1899297920 wait complete. 2023-10-08T09:54:04.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T09:53:56.599415+0000', 'modified': '2023-10-08T09:54:01.623120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6839/1665352653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 1665352653}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 1665352653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-08T09:54:04.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T09:54:04.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 -- 172.21.15.97:0/1416611885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fff70 msgr2=0x7feab41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1416611885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fff70 0x7feab41003f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feab0009990 tx=0x7feab0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 -- 172.21.15.97:0/1416611885 shutdown_connections 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1416611885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab4100930 0x7feab4107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1416611885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab40fee20 0x7feab40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1416611885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fff70 0x7feab41003f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 -- 172.21.15.97:0/1416611885 >> 172.21.15.97:0/1416611885 conn(0x7feab40fa390 msgr2=0x7feab40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 -- 172.21.15.97:0/1416611885 shutdown_connections 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.188+0000 7feabc2f6700 1 -- 172.21.15.97:0/1416611885 wait complete. 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 Processor -- start 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 -- start start 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab40fff70 0x7feab4194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab4100930 0x7feab419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab419c020 con 0x7feab40fee20 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab419c160 con 0x7feab40fff70 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feabc2f6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab419c2f0 con 0x7feab4100930 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55242/0 (socket says 172.21.15.97:55242) 2023-10-08T09:54:04.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 -- 172.21.15.97:0/1076965430 learned_addr learned my addr 172.21.15.97:0/1076965430 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 -- 172.21.15.97:0/1076965430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab4100930 msgr2=0x7feab419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab4100930 0x7feab419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feab9891700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab40fff70 0x7feab4194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 -- 172.21.15.97:0/1076965430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab40fff70 msgr2=0x7feab4194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab40fff70 0x7feab4194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.189+0000 7feaba092700 1 -- 172.21.15.97:0/1076965430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00095f0 con 0x7feab40fee20 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feaba092700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feaa400c920 tx=0x7feaa400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feaab7fe700 1 -- 172.21.15.97:0/1076965430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4007b00 con 0x7feab40fee20 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feaab7fe700 1 -- 172.21.15.97:0/1076965430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa4004cf0 con 0x7feab40fee20 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab419c5d0 con 0x7feab40fee20 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feaab7fe700 1 -- 172.21.15.97:0/1076965430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4005610 con 0x7feab40fee20 2023-10-08T09:54:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.190+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab419cb30 con 0x7feab40fee20 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.192+0000 7feaab7fe700 1 -- 172.21.15.97:0/1076965430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feaa4004750 con 0x7feab40fee20 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.192+0000 7feaab7fe700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 0x7feaa00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.192+0000 7feaab7fe700 1 -- 172.21.15.97:0/1076965430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feaa40cce30 con 0x7feab40fee20 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.192+0000 7feab9891700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 0x7feaa00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.193+0000 7feab9891700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 0x7feaa00a39c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feab00037f0 tx=0x7feab0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.193+0000 7feac84dfb80 1 Processor -- start 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.193+0000 7feac84dfb80 1 -- start start 2023-10-08T09:54:04.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.193+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 0x556dd4b45430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b45e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feac84dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4aaa410 con 0x556dd4b44220 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feac84dfb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a9bbd0 con 0x556dd4b44b80 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feac84dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a994f0 con 0x556dd4b459c0 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55250/0 (socket says 172.21.15.97:55250) 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 -- 172.21.15.97:0/4206757051 learned_addr learned my addr 172.21.15.97:0/4206757051 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 -- 172.21.15.97:0/4206757051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 msgr2=0x556dd4b45e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feab9891700 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 0x556dd4b45430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b45e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 -- 172.21.15.97:0/4206757051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 msgr2=0x556dd4b45430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 0x556dd4b45430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 -- 172.21.15.97:0/4206757051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd48bfd90 con 0x556dd4b44220 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feab9891700 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 0x556dd4b45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaba092700 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feaa4099ef0 tx=0x7feaa40050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9996f884a703497 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaa8d82700 1 -- 172.21.15.97:0/4206757051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa409e0b0 con 0x556dd4b44220 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaa8d82700 1 -- 172.21.15.97:0/4206757051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa40a22f0 con 0x556dd4b44220 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.194+0000 7feaa8d82700 1 -- 172.21.15.97:0/4206757051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4096670 con 0x556dd4b44220 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- 172.21.15.97:0/4206757051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 msgr2=0x556dd4b44640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feaa4099ef0 tx=0x7feaa40050b0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- 172.21.15.97:0/4206757051 shutdown_connections 2023-10-08T09:54:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b45e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44220 0x556dd4b44640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 --2- 172.21.15.97:0/4206757051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44b80 0x556dd4b45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- 172.21.15.97:0/4206757051 >> 172.21.15.97:0/4206757051 conn(0x556dd4ab6210 msgr2=0x556dd4b3cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- 172.21.15.97:0/4206757051 shutdown_connections 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- 172.21.15.97:0/4206757051 wait complete. 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 -- start start 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.195+0000 7feac84dfb80 1 Processor -- start 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44b80 0x556dd4b4e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a94de0 con 0x556dd4b459c0 2023-10-08T09:54:04.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a316c0 con 0x556dd4b44220 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a31360 con 0x556dd4b44b80 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 10 client.0 _ll_register_callbacks cb 0x556dd488e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55260/0 (socket says 172.21.15.97:55260) 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 -- 172.21.15.97:0/3447622399 learned_addr learned my addr 172.21.15.97:0/3447622399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 -- 172.21.15.97:0/3447622399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44b80 msgr2=0x556dd4b4e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba092700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44b80 0x556dd4b4e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 -- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 msgr2=0x556dd4b4e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b4e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 -- 172.21.15.97:0/3447622399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab419cb30 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feaba893700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feaac007ec0 tx=0x7feaac00d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c004d79dfbbf417a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7fea9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac017070 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feaac00ae40 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dd4a5ce60 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaac00f040 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 10 client.14886 fetch_fsmap learned FSMap version 17 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.196+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x556dd48a8c10 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556dd4a5ce60 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7feaac01a020 con 0x556dd4b459c0 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 10 client.14886 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 10 client.14886 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:54:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 20 client.14886 populate_metadata read hostname 'smithi097' 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4b59c60 0x556dd4b586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 4 client.14886 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dd4a8ddc0 con 0x556dd4b59c60 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.197+0000 7feaba092700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4b59c60 0x556dd4b586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7fea9dffb700 10 client.14886 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feaba092700 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4b59c60 0x556dd4b586b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80d419a1c6a2235 server_cookie=65d525f202097ea7 in_seq=0 out_seq=0 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7fea9dffb700 1 -- 172.21.15.97:0/3447622399 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556dd4a8ddc0 con 0x556dd4b59c60 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7fea9dffb700 10 client.14886 handle_command_reply: tid=0 2023-10-08T09:54:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 client.14886 shutdown 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 msgr2=0x556dd4b555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feaac007ec0 tx=0x7feaac00d3a0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 shutdown_connections 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b459c0 0x556dd4b555a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4b59c60 0x556dd4b586b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 --2- 172.21.15.97:0/3447622399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44b80 0x556dd4b4e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7fea9dffb700 0 client.14886 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 >> 172.21.15.97:0/3447622399 conn(0x556dd4ab6210 msgr2=0x556dd4b3c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 shutdown_connections 2023-10-08T09:54:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.198+0000 7feac84dfb80 1 -- 172.21.15.97:0/3447622399 wait complete. 2023-10-08T09:54:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.199+0000 7feac84dfb80 20 client.14886 trim_cache size 0 max 16384 2023-10-08T09:54:04.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 Processor -- start 2023-10-08T09:54:04.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 -- start start 2023-10-08T09:54:04.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44220 0x556dd4b4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 0x556dd4b4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd49158d0 con 0x556dd4b44b80 2023-10-08T09:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4b55a00 con 0x556dd4b459c0 2023-10-08T09:54:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feac84dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4a8ddc0 con 0x556dd4b44220 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55266/0 (socket says 172.21.15.97:55266) 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 -- 172.21.15.97:0/1918578013 learned_addr learned my addr 172.21.15.97:0/1918578013 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feaba893700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 0x556dd4b4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feaba092700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44220 0x556dd4b4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 -- 172.21.15.97:0/1918578013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44220 msgr2=0x556dd4b4d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44220 0x556dd4b4d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 -- 172.21.15.97:0/1918578013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 msgr2=0x556dd4b4e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 0x556dd4b4e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feab9891700 1 -- 172.21.15.97:0/1918578013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd48e6200 con 0x556dd4b44b80 2023-10-08T09:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.216+0000 7feaba893700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 0x556dd4b4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feab9891700 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feab0005170 tx=0x7feab0019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f5d83790b8a596 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feaa9583700 1 -- 172.21.15.97:0/1918578013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab002b070 con 0x556dd4b44b80 2023-10-08T09:54:04.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feaa9583700 1 -- 172.21.15.97:0/1918578013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab001d040 con 0x556dd4b44b80 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feaa9583700 1 -- 172.21.15.97:0/1918578013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0004d90 con 0x556dd4b44b80 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- 172.21.15.97:0/1918578013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 msgr2=0x556dd4b554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feab0005170 tx=0x7feab0019050 comp rx=0 tx=0).stop 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- 172.21.15.97:0/1918578013 shutdown_connections 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b459c0 0x556dd4b4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b44220 0x556dd4b4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 --2- 172.21.15.97:0/1918578013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4b44b80 0x556dd4b554c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- 172.21.15.97:0/1918578013 >> 172.21.15.97:0/1918578013 conn(0x556dd4ab6210 msgr2=0x556dd4b4bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- 172.21.15.97:0/1918578013 shutdown_connections 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- 172.21.15.97:0/1918578013 wait complete. 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.217+0000 7feac84dfb80 1 -- start start 2023-10-08T09:54:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 Processor -- start 2023-10-08T09:54:04.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4badf70 con 0x556dd4bab640 2023-10-08T09:54:04.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4bae0b0 con 0x556dd4b44220 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd4bae270 con 0x556dd4b459c0 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feac84dfb80 10 client.0 _ll_register_callbacks cb 0x7feaac007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55274/0 (socket says 172.21.15.97:55274) 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 -- 172.21.15.97:0/719604201 learned_addr learned my addr 172.21.15.97:0/719604201 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:04.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba092700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feab9891700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 -- 172.21.15.97:0/719604201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 msgr2=0x556dd4b52a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b52a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 -- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 msgr2=0x556dd4b524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:54:04.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 -- 172.21.15.97:0/719604201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab4107b80 con 0x556dd4bab640 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feab9891700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7feaba893700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaac000f80 tx=0x7feaac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61432f8afec18b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.218+0000 7fea9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00f070 con 0x556dd4bab640 2023-10-08T09:54:04.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feaac00ae40 con 0x556dd4bab640 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa40a22f0 con 0x556dd4bab640 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaac017070 con 0x556dd4bab640 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 10 client.14892 fetch_fsmap learned FSMap version 17 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x556dd48be260 con 0x556dd4bab640 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa40a22f0 con 0x556dd4bab640 2023-10-08T09:54:04.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7feaac00d0c0 con 0x556dd4bab640 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 10 client.14892 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 10 client.14892 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 20 client.14892 populate_metadata read hostname 'smithi097' 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4bb35d0 0x556dd4bafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 4 client.14892 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:54:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556dd4a8dae0 con 0x556dd4bb35d0 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.219+0000 7feaba092700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4bb35d0 0x556dd4bafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7fea9dffb700 10 client.14892 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feaba092700 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4bb35d0 0x556dd4bafa10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbe38a50fff66e server_cookie=806c5765900c831d in_seq=0 out_seq=0 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7fea9dffb700 1 -- 172.21.15.97:0/719604201 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556dd4a8dae0 con 0x556dd4bb35d0 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7fea9dffb700 10 client.14892 handle_command_reply: tid=0 2023-10-08T09:54:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 client.14892 shutdown 2023-10-08T09:54:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 msgr2=0x556dd4bada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaac000f80 tx=0x7feaac009da0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 shutdown_connections 2023-10-08T09:54:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dd4bab640 0x556dd4bada30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd4b44220 0x556dd4b524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x556dd4bb35d0 0x556dd4bafa10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 --2- 172.21.15.97:0/719604201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556dd4b459c0 0x556dd4b52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7fea9dffb700 0 client.14892 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 >> 172.21.15.97:0/719604201 conn(0x556dd4ab6210 msgr2=0x556dd4b4d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 shutdown_connections 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 1 -- 172.21.15.97:0/719604201 wait complete. 2023-10-08T09:54:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.220+0000 7feac84dfb80 20 client.14892 trim_cache size 0 max 16384 2023-10-08T09:54:04.240 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:54:04.240 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:54:04.240 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-10-08T09:54:04.240 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24566, 2023-10-08T09:54:04.241 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:54:04.241 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:54:04.241 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:54:04.241 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 7.0206515539999996, 2023-10-08T09:54:04.241 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 17, 2023-10-08T09:54:04.242 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 85, 2023-10-08T09:54:04.242 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:54:04.242 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.166700913 2023-10-08T09:54:04.242 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:54:04.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 msgr2=0x7feaa00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 0x7feaa00a39c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feab00037f0 tx=0x7feab0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 msgr2=0x7feab41944f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:04.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feaa400c920 tx=0x7feaa400cce0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 shutdown_connections 2023-10-08T09:54:04.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1500 0x7feaa00a39c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab4100930 0x7feab419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab40fee20 0x7feab41944f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 --2- 172.21.15.97:0/1076965430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab40fff70 0x7feab4194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:04.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 >> 172.21.15.97:0/1076965430 conn(0x7feab40fa390 msgr2=0x7feab41033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:04.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 shutdown_connections 2023-10-08T09:54:04.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:04.221+0000 7feabc2f6700 1 -- 172.21.15.97:0/1076965430 wait complete. 2023-10-08T09:54:04.247 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T09:54:04.247 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:04.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T09:54:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:04 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2044977536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:04 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/509361922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:04 smithi097 ceph-mon[99491]: pgmap v264: 129 pgs: 12 creating+peering, 8 unknown, 109 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-10-08T09:54:04.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:04 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1899297920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:04 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2044977536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:04 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/509361922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:04 smithi186 ceph-mon[104393]: pgmap v264: 129 pgs: 12 creating+peering, 8 unknown, 109 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-10-08T09:54:04.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:04 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1899297920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:04 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2044977536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:04 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/509361922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:04 smithi178 ceph-mon[103156]: pgmap v264: 129 pgs: 12 creating+peering, 8 unknown, 109 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 9 op/s 2023-10-08T09:54:04.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:04 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1899297920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:05.373 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:05 smithi097 ceph-mon[99491]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:54:05.373 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:05 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T09:54:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:05 smithi186 ceph-mon[104393]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:54:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:05 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T09:54:05.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:05 smithi178 ceph-mon[103156]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:54:05.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:05 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T09:54:06.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/1880060183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0100c60 msgr2=0x7f1aa010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:06.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/1880060183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0100c60 0x7f1aa010c5c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800b0a0 tx=0x7f1a9800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/1880060183 shutdown_connections 2023-10-08T09:54:06.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/1880060183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0100c60 0x7f1aa010c5c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/1880060183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0104720 0x7f1aa0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/1880060183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 0x7f1aa0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/1880060183 >> 172.21.15.97:0/1880060183 conn(0x7f1aa0074a10 msgr2=0x7f1aa0074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:06.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/1880060183 shutdown_connections 2023-10-08T09:54:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.130+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/1880060183 wait complete. 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.131+0000 7f1aa5e4d700 1 Processor -- start 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.131+0000 7f1aa5e4d700 1 -- start start 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 0x7f1aa0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0100c60 0x7f1aa01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa01989a0 con 0x7f1aa0104720 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa01a3160 con 0x7f1aa0100c60 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa01a32a0 con 0x7f1aa01002a0 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55288/0 (socket says 172.21.15.97:55288) 2023-10-08T09:54:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 -- 172.21.15.97:0/2121044470 learned_addr learned my addr 172.21.15.97:0/2121044470 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9f7fe700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 0x7f1aa0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9effd700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0100c60 0x7f1aa01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 -- 172.21.15.97:0/2121044470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 msgr2=0x7f1aa0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 0x7f1aa0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 -- 172.21.15.97:0/2121044470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0100c60 msgr2=0x7f1aa01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0100c60 0x7f1aa01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 -- 172.21.15.97:0/2121044470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98009d00 con 0x7f1aa0104720 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9ffff700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800b070 tx=0x7f1a98007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800e040 con 0x7f1aa0104720 2023-10-08T09:54:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa01a34c0 con 0x7f1aa0104720 2023-10-08T09:54:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a98009440 con 0x7f1aa0104720 2023-10-08T09:54:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9801b6b0 con 0x7f1aa0104720 2023-10-08T09:54:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.132+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01a3980 con 0x7f1aa0104720 2023-10-08T09:54:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.133+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0107d10 con 0x7f1aa0104720 2023-10-08T09:54:06.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.134+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1a9801d480 con 0x7f1aa0104720 2023-10-08T09:54:06.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.135+0000 7f1a9cff9700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 0x7f1a880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:06.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.135+0000 7f1a9f7fe700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 0x7f1a880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:06.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.135+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1a980ce4d0 con 0x7f1aa0104720 2023-10-08T09:54:06.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.135+0000 7f1a9f7fe700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 0x7f1a880a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005950 tx=0x7f1a940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.139+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a980979d0 con 0x7f1aa0104720 2023-10-08T09:54:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.371+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1aa00ff750 con 0x7f1aa0104720 2023-10-08T09:54:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.371+0000 7f1a9cff9700 1 -- 172.21.15.97:0/2121044470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1a9801de60 con 0x7f1aa0104720 2023-10-08T09:54:06.372 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 msgr2=0x7f1a880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 0x7f1a880a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005950 tx=0x7f1a940058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 msgr2=0x7f1aa01a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800b070 tx=0x7f1a98007920 comp rx=0 tx=0).stop 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 shutdown_connections 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1a880a1480 0x7f1a880a3940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aa0104720 0x7f1aa01a2c20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1aa01002a0 0x7f1aa0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 --2- 172.21.15.97:0/2121044470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa0100c60 0x7f1aa01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 >> 172.21.15.97:0/2121044470 conn(0x7f1aa0074a10 msgr2=0x7f1aa00fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 shutdown_connections 2023-10-08T09:54:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:06.373+0000 7f1aa5e4d700 1 -- 172.21.15.97:0/2121044470 wait complete. 2023-10-08T09:54:06.517 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:06 smithi097 ceph-mon[99491]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2023-10-08T09:54:06.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:06 smithi186 ceph-mon[104393]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2023-10-08T09:54:06.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:06 smithi178 ceph-mon[103156]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2023-10-08T09:54:06.756 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T09:54:06.756 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T09:54:06.756 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-10-08T09:54:06.756 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-08T09:54:06.757 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:06.757 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T09:54:06.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T09:54:06.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T09:54:06.757 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T09:54:06.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:06.757 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-10-08T09:54:06.776 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T09:54:06.776 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T09:54:06.776 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:54:06.777 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T09:54:06.778 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41919sec preferred_lft 41919sec 2023-10-08T09:54:06.778 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7efb:78c9:efd1:4682/64 scope link noprefixroute 2023-10-08T09:54:06.778 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:54:06.778 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T09:54:06.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:06.778 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:54:06.779 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:54:06.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2023-10-08T09:54:06.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2023-10-08T09:54:06.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2023-10-08T09:54:06.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T09:54:06.780 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:54:06.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:06.948 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T09:54:06.976 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-10-08T09:54:06.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:06.977 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-10-08T09:54:07.046 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-10-08T09:54:07.046 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T09:54:07.046 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T09:54:07.046 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T09:54:07.047 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T09:54:07.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.047 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:54:07.047 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:54:07.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T09:54:07.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T09:54:07.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T09:54:07.048 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:54:07.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.284 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-10-08T09:54:07.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.301 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-10-08T09:54:07.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.358 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:54:07.358 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:54:07.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:54:07.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-08T09:54:07.359 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:54:07.436 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:07 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2121044470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:54:07.452 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-08T09:54:07.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-08T09:54:07.453 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:54:07.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:07 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2121044470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:54:07.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:07.705 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:54:07.705 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:54:07.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-10-08T09:54:07.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T09:54:07.705 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:54:07.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:07 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2121044470' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:54:07.804 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-08T09:54:07.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:54:07.804 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:07.821 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T09:54:07.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:54:07.821 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:07.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-10-08T09:54:07.946 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T09:54:08.004 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2023-10-08T09:54:08.004 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/systemd 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/perf_event 2023-10-08T09:54:08.005 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/devices 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/memory 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/cpuset 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/rdma 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/blkio 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/freezer 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/pids 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2023-10-08T09:54:08.006 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2023-10-08T09:54:08.007 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/5a643881f72a24e8b4f3fd71b2a4c349060ae46c11d773569c7e761c4f830528/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/85af0a6bac16745811e157f7d7e0b1bf9f9ad7afbac4107cf89449bcd95433c1/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/5431c2514ddcb6d629b885f71b31e929e5fe700da2781bb78e7bb9485231f084/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/35c687c932f052f75bff2684fa6bf5a9380352b8ac6083936958d1064924d9be/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/81e7cb7de7f5be8d8cd46ab774daa5bf9d12a101f768460997d3ff8a01dac0e6/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/502a3e9f65ec51771d7bccd809738a4bd2f4d1a48aa9b2618b1db76bd56f5532/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/6880309c8f93421e16ba220e603193ef13add9d6eba1cac93b80154ee0302fc8/merged 2023-10-08T09:54:08.008 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/71ddf0ec124060da9168cfec1e71ff105be210c19acb8988610f8c5ccd19a583/merged 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/bc8e8daa2a3054f87b3432cc64e376c6f68827430299b955ac15e698cb04aba2/merged 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/6897173f511c0308d8c287bc9a09dd8beb4835ca44c16ef9cf5f551f6901f730/merged 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:54:08.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.009 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-10-08T09:54:08.061 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T09:54:08.061 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-08T09:54:08.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-10-08T09:54:08.129 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T09:54:08.186 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2023-10-08T09:54:08.186 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/systemd 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/perf_event 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/devices 2023-10-08T09:54:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/memory 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/cpuset 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/rdma 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/blkio 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/freezer 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup/pids 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2023-10-08T09:54:08.188 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/5a643881f72a24e8b4f3fd71b2a4c349060ae46c11d773569c7e761c4f830528/merged 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/85af0a6bac16745811e157f7d7e0b1bf9f9ad7afbac4107cf89449bcd95433c1/merged 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/0 2023-10-08T09:54:08.189 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/5431c2514ddcb6d629b885f71b31e929e5fe700da2781bb78e7bb9485231f084/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/35c687c932f052f75bff2684fa6bf5a9380352b8ac6083936958d1064924d9be/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/81e7cb7de7f5be8d8cd46ab774daa5bf9d12a101f768460997d3ff8a01dac0e6/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/502a3e9f65ec51771d7bccd809738a4bd2f4d1a48aa9b2618b1db76bd56f5532/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/6880309c8f93421e16ba220e603193ef13add9d6eba1cac93b80154ee0302fc8/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/71ddf0ec124060da9168cfec1e71ff105be210c19acb8988610f8c5ccd19a583/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/bc8e8daa2a3054f87b3432cc64e376c6f68827430299b955ac15e698cb04aba2/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/containers/storage/overlay/6897173f511c0308d8c287bc9a09dd8beb4835ca44c16ef9cf5f551f6901f730/merged 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:54:08.190 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:54:08.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.191 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-10-08T09:54:08.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:2023-10-08T09:54:08.220+0000 7fa01dd985c0 -1 init, newargv = 0x55e6c28727e0 newargc=15 2023-10-08T09:54:08.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[158444]: starting ceph client 2023-10-08T09:54:08.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[158444]: starting fuse 2023-10-08T09:54:08.237 INFO:teuthology.orchestra.run.smithi097.stdout:124 2023-10-08T09:54:08.238 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-10-08T09:54:08.238 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T09:54:08.239 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.admin")) 2023-10-08T09:54:08.240 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-10-08T09:54:08.361 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.admin.158444.asok 2023-10-08T09:54:08.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-08T09:54:08.371 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.158444.asok 2023-10-08T09:54:08.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.158444.asok status 2023-10-08T09:54:08.434 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:08 smithi097 ceph-mon[99491]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 669 B/s wr, 4 op/s 2023-10-08T09:54:08.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:08 smithi186 ceph-mon[104393]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 669 B/s wr, 4 op/s 2023-10-08T09:54:08.526 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:54:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "7b3143bae3b6870a749bfb9810ed6e7d0e89d7de", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "admin", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "158444", 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-10-08T09:54:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 14904, 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 14904 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3810101069 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:54:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3810101069 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.14904 192.168.0.1:0/3810101069", 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/3810101069", 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 17, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 85, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-10-08T09:54:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-10-08T09:54:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-10-08T09:54:08.530 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:54:08.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.537 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-10-08T09:54:08.537 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:08.594 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-10-08T09:54:08.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:08.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:08.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:08.663 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-08T09:54:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:08 smithi178 ceph-mon[103156]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 669 B/s wr, 4 op/s 2023-10-08T09:54:08.764 INFO:teuthology.orchestra.run.smithi097.stdout:# file: . 2023-10-08T09:54:08.764 INFO:teuthology.orchestra.run.smithi097.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-08T09:54:08.764 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:08.764 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:54:08.765 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:54:08.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 -- 172.21.15.97:0/1966669545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f01011d0 msgr2=0x7f80f01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1966669545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f01011d0 0x7f80f01035c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 -- 172.21.15.97:0/1966669545 shutdown_connections 2023-10-08T09:54:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1966669545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f0106430 0x7f80f0108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1966669545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f01011d0 0x7f80f01035c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1966669545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0103b00 0x7f80f0105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 -- 172.21.15.97:0/1966669545 >> 172.21.15.97:0/1966669545 conn(0x7f80f00fab40 msgr2=0x7f80f00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 -- 172.21.15.97:0/1966669545 shutdown_connections 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.943+0000 7f80f6a53700 1 -- 172.21.15.97:0/1966669545 wait complete. 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 Processor -- start 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 -- start start 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f01011d0 0x7f80f0100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 0x7f80f00ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00ffe60 con 0x7f80f0103b00 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00fffd0 con 0x7f80f0106430 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f6a53700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01a0640 con 0x7f80f01011d0 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55312/0 (socket says 172.21.15.97:55312) 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 -- 172.21.15.97:0/1861448635 learned_addr learned my addr 172.21.15.97:0/1861448635 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f4ff0700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 0x7f80f00ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80effff700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f01011d0 0x7f80f0100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 -- 172.21.15.97:0/1861448635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f01011d0 msgr2=0x7f80f0100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f01011d0 0x7f80f0100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 -- 172.21.15.97:0/1861448635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 msgr2=0x7f80f00ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 0x7f80f00ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 -- 172.21.15.97:0/1861448635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80095f0 con 0x7f80f0103b00 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80f4ff0700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 0x7f80f00ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.944+0000 7f80ef7fe700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c9b0 tx=0x7f80e000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.945+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0007aa0 con 0x7f80f0103b00 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.945+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e0007c00 con 0x7f80f0103b00 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.945+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f01a08d0 con 0x7f80f0103b00 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.945+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0018660 con 0x7f80f0103b00 2023-10-08T09:54:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.945+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f01a0de0 con 0x7f80f0103b00 2023-10-08T09:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.946+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00fc6c0 con 0x7f80f0103b00 2023-10-08T09:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.947+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80e000f450 con 0x7f80f0103b00 2023-10-08T09:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.947+0000 7f80ed7fa700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 0x7f80dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.947+0000 7f80effff700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 0x7f80dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.947+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f80e00cd210 con 0x7f80f0103b00 2023-10-08T09:54:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.948+0000 7f80effff700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 0x7f80dc0a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b5c0 tx=0x7f80d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:08.951+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f80e00966b0 con 0x7f80f0103b00 2023-10-08T09:54:09.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.144+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80f0066bc0 con 0x7f80f0103b00 2023-10-08T09:54:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.146+0000 7f80ed7fa700 1 -- 172.21.15.97:0/1861448635 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f80e0099cd0 con 0x7f80f0103b00 2023-10-08T09:54:09.147 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:09.147 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 msgr2=0x7f80dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 0x7f80dc0a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b5c0 tx=0x7f80d8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 msgr2=0x7f80f00ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c9b0 tx=0x7f80e000cd70 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 shutdown_connections 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f80dc0a1480 0x7f80dc0a3940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80f0106430 0x7f80f00ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80f01011d0 0x7f80f0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.148+0000 7f80f6a53700 1 --2- 172.21.15.97:0/1861448635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f0103b00 0x7f80f00ff180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.149+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 >> 172.21.15.97:0/1861448635 conn(0x7f80f00fab40 msgr2=0x7f80f0101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.149+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 shutdown_connections 2023-10-08T09:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.149+0000 7f80f6a53700 1 -- 172.21.15.97:0/1861448635 wait complete. 2023-10-08T09:54:09.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:54:09.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:54:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 -- 172.21.15.97:0/464381227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c00ff6a0 msgr2=0x7f06c00ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 --2- 172.21.15.97:0/464381227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c00ff6a0 0x7f06c00ffac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f06b40099c0 tx=0x7f06b4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 -- 172.21.15.97:0/464381227 shutdown_connections 2023-10-08T09:54:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 --2- 172.21.15.97:0/464381227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0101230 0x7f06c0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 --2- 172.21.15.97:0/464381227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c00ff6a0 0x7f06c00ffac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 --2- 172.21.15.97:0/464381227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c0100890 0x7f06c0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 -- 172.21.15.97:0/464381227 >> 172.21.15.97:0/464381227 conn(0x7f06c00fac10 msgr2=0x7f06c00fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 -- 172.21.15.97:0/464381227 shutdown_connections 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.287+0000 7f06c5178700 1 -- 172.21.15.97:0/464381227 wait complete. 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 Processor -- start 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 -- start start 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 0x7f06c01955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0100890 0x7f06c0195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c019d0e0 con 0x7f06c0101230 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c019d220 con 0x7f06c00ff6a0 2023-10-08T09:54:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06c5178700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c019d3b0 con 0x7f06c0100890 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55328/0 (socket says 172.21.15.97:55328) 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 -- 172.21.15.97:0/3760302411 learned_addr learned my addr 172.21.15.97:0/3760302411 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06be59c700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0100890 0x7f06c0195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bed9d700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 0x7f06c01955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 -- 172.21.15.97:0/3760302411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0100890 msgr2=0x7f06c0195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0100890 0x7f06c0195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 -- 172.21.15.97:0/3760302411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 msgr2=0x7f06c01955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 0x7f06c01955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.288+0000 7f06bf59e700 1 -- 172.21.15.97:0/3760302411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40095f0 con 0x7f06c0101230 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06bed9d700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 0x7f06c01955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06bf59e700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b8007ee0 tx=0x7f06b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8017070 con 0x7f06c0101230 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c019d690 con 0x7f06c0101230 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06b800f040 con 0x7f06c0101230 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8005740 con 0x7f06c0101230 2023-10-08T09:54:09.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.289+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c019dba0 con 0x7f06c0101230 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.290+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0106ad0 con 0x7f06c0101230 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.291+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f06b800dc60 con 0x7f06c0101230 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.291+0000 7f06a7fff700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 0x7f06a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.292+0000 7f06bed9d700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 0x7f06a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.292+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06b80cc6e0 con 0x7f06c0101230 2023-10-08T09:54:09.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.292+0000 7f06bed9d700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 0x7f06a80a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f06b400b580 tx=0x7f06b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.296+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f06b8095af0 con 0x7f06c0101230 2023-10-08T09:54:09.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:09 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1861448635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.488+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06c01064b0 con 0x7f06c0101230 2023-10-08T09:54:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.488+0000 7f06a7fff700 1 -- 172.21.15.97:0/3760302411 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06b8099110 con 0x7f06c0101230 2023-10-08T09:54:09.489 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 msgr2=0x7f06a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 0x7f06a80a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f06b400b580 tx=0x7f06b4005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 msgr2=0x7f06c019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06b8007ee0 tx=0x7f06b800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 shutdown_connections 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f06a80a1480 0x7f06a80a3940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c0101230 0x7f06c019cb10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c00ff6a0 0x7f06c01955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 --2- 172.21.15.97:0/3760302411 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0100890 0x7f06c0195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 >> 172.21.15.97:0/3760302411 conn(0x7f06c00fac10 msgr2=0x7f06c0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 shutdown_connections 2023-10-08T09:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.491+0000 7f06c5178700 1 -- 172.21.15.97:0/3760302411 wait complete. 2023-10-08T09:54:09.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:54:09.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:09 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1861448635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- 172.21.15.97:0/4051375911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 msgr2=0x7ff3e8107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 --2- 172.21.15.97:0/4051375911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e8107250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000b0a0 tx=0x7ff3e000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- 172.21.15.97:0/4051375911 shutdown_connections 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 --2- 172.21.15.97:0/4051375911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e8107250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 --2- 172.21.15.97:0/4051375911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fee20 0x7ff3e80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 --2- 172.21.15.97:0/4051375911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fff70 0x7ff3e81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- 172.21.15.97:0/4051375911 >> 172.21.15.97:0/4051375911 conn(0x7ff3e80fa390 msgr2=0x7ff3e80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- 172.21.15.97:0/4051375911 shutdown_connections 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- 172.21.15.97:0/4051375911 wait complete. 2023-10-08T09:54:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 Processor -- start 2023-10-08T09:54:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.627+0000 7ff3edd87700 1 -- start start 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fee20 0x7ff3e81086c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 0x7ff3e819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e3c0 con 0x7ff3e8100930 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e530 con 0x7ff3e80fff70 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e6f0 con 0x7ff3e80fee20 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55342/0 (socket says 172.21.15.97:55342) 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 -- 172.21.15.97:0/3214970703 learned_addr learned my addr 172.21.15.97:0/3214970703 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:09.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e6ffd700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 0x7ff3e819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e77fe700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fee20 0x7ff3e81086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 -- 172.21.15.97:0/3214970703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fee20 msgr2=0x7ff3e81086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fee20 0x7ff3e81086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 -- 172.21.15.97:0/3214970703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 msgr2=0x7ff3e819b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 0x7ff3e819b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 -- 172.21.15.97:0/3214970703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009d00 con 0x7ff3e8100930 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e6ffd700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 0x7ff3e819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e7fff700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0006070 tx=0x7ff3e000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000e040 con 0x7ff3e8100930 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3e0004e40 con 0x7ff3e8100930 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e819e910 con 0x7ff3e8100930 2023-10-08T09:54:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e001b810 con 0x7ff3e8100930 2023-10-08T09:54:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.628+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e819ed90 con 0x7ff3e8100930 2023-10-08T09:54:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.629+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e81061c0 con 0x7ff3e8100930 2023-10-08T09:54:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.630+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff3e001b970 con 0x7ff3e8100930 2023-10-08T09:54:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.631+0000 7ff3e4ff9700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 0x7ff3d00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.631+0000 7ff3e77fe700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 0x7ff3d00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.631+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff3e00ce9a0 con 0x7ff3e8100930 2023-10-08T09:54:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.631+0000 7ff3e77fe700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 0x7ff3d00a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b30 tx=0x7ff3dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.635+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3e0097ea0 con 0x7ff3e8100930 2023-10-08T09:54:09.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:09 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1861448635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.827+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3e8105ba0 con 0x7ff3e8100930 2023-10-08T09:54:09.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.828+0000 7ff3e4ff9700 1 -- 172.21.15.97:0/3214970703 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3e009b4c0 con 0x7ff3e8100930 2023-10-08T09:54:09.829 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 msgr2=0x7ff3d00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 0x7ff3d00a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b30 tx=0x7ff3dc009260 comp rx=0 tx=0).stop 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 msgr2=0x7ff3e819dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0006070 tx=0x7ff3e000bdb0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 shutdown_connections 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff3d00a1480 0x7ff3d00a3940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e8100930 0x7ff3e819dbd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e80fee20 0x7ff3e81086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 --2- 172.21.15.97:0/3214970703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3e80fff70 0x7ff3e819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 >> 172.21.15.97:0/3214970703 conn(0x7ff3e80fa390 msgr2=0x7ff3e8103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 shutdown_connections 2023-10-08T09:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.830+0000 7ff3edd87700 1 -- 172.21.15.97:0/3214970703 wait complete. 2023-10-08T09:54:09.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:54:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 -- 172.21.15.97:0/1976571161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe181008a0 msgr2=0x7fbe18100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1976571161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe181008a0 0x7fbe18100d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe140099c0 tx=0x7fbe14009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 -- 172.21.15.97:0/1976571161 shutdown_connections 2023-10-08T09:54:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1976571161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe18101240 0x7fbe18107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1976571161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe180ff6b0 0x7fbe180ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1976571161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe181008a0 0x7fbe18100d00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 -- 172.21.15.97:0/1976571161 >> 172.21.15.97:0/1976571161 conn(0x7fbe180fac40 msgr2=0x7fbe180fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 -- 172.21.15.97:0/1976571161 shutdown_connections 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.965+0000 7fbe2008c700 1 -- 172.21.15.97:0/1976571161 wait complete. 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 Processor -- start 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 -- start start 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 0x7fbe180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe181008a0 0x7fbe1819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819e1d0 con 0x7fbe18101240 2023-10-08T09:54:09.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819e340 con 0x7fbe180ff6b0 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe2008c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819e500 con 0x7fbe181008a0 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55356/0 (socket says 172.21.15.97:55356) 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 -- 172.21.15.97:0/1320135640 learned_addr learned my addr 172.21.15.97:0/1320135640 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1d627700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe181008a0 0x7fbe1819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1de28700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 0x7fbe180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 -- 172.21.15.97:0/1320135640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe181008a0 msgr2=0x7fbe1819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe181008a0 0x7fbe1819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 -- 172.21.15.97:0/1320135640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 msgr2=0x7fbe180697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 0x7fbe180697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 -- 172.21.15.97:0/1320135640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140095f0 con 0x7fbe18101240 2023-10-08T09:54:09.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1de28700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 0x7fbe180697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe1e629700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbe10007ee0 tx=0x7fbe1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.966+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10017070 con 0x7fbe18101240 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.967+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1819e780 con 0x7fbe18101240 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.967+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe1000f040 con 0x7fbe18101240 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.967+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10005740 con 0x7fbe18101240 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.967+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1819ec10 con 0x7fbe18101240 2023-10-08T09:54:09.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.967+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe18106b30 con 0x7fbe18101240 2023-10-08T09:54:09.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.969+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbe100058a0 con 0x7fbe18101240 2023-10-08T09:54:09.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.969+0000 7fbe0effd700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 0x7fbe040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:09.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.969+0000 7fbe1de28700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 0x7fbe040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:09.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.969+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbe100ccdf0 con 0x7fbe18101240 2023-10-08T09:54:09.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.970+0000 7fbe1de28700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 0x7fbe040a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005fd0 tx=0x7fbe08005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:09.973+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbe10096260 con 0x7fbe18101240 2023-10-08T09:54:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.164+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe1810b020 con 0x7fbe18101240 2023-10-08T09:54:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.165+0000 7fbe0effd700 1 -- 172.21.15.97:0/1320135640 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe10099880 con 0x7fbe18101240 2023-10-08T09:54:10.166 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:54:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 msgr2=0x7fbe040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 0x7fbe040a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe08005fd0 tx=0x7fbe08005e40 comp rx=0 tx=0).stop 2023-10-08T09:54:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 msgr2=0x7fbe1819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbe10007ee0 tx=0x7fbe1000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 shutdown_connections 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbe040a1480 0x7fbe040a3940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe18101240 0x7fbe1819d9e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe180ff6b0 0x7fbe180697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 --2- 172.21.15.97:0/1320135640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe181008a0 0x7fbe1819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 >> 172.21.15.97:0/1320135640 conn(0x7fbe180fac40 msgr2=0x7fbe18103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 shutdown_connections 2023-10-08T09:54:10.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.167+0000 7fbe2008c700 1 -- 172.21.15.97:0/1320135640 wait complete. 2023-10-08T09:54:10.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:54:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 -- 172.21.15.97:0/3557681534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a581008a0 msgr2=0x7f9a58100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 --2- 172.21.15.97:0/3557681534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a581008a0 0x7f9a58100d00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9a480099c0 tx=0x7f9a48009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 -- 172.21.15.97:0/3557681534 shutdown_connections 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 --2- 172.21.15.97:0/3557681534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 0x7f9a58107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 --2- 172.21.15.97:0/3557681534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a580ff6b0 0x7f9a580ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 --2- 172.21.15.97:0/3557681534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a581008a0 0x7f9a58100d00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 -- 172.21.15.97:0/3557681534 >> 172.21.15.97:0/3557681534 conn(0x7f9a580fac40 msgr2=0x7f9a580fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 -- 172.21.15.97:0/3557681534 shutdown_connections 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.304+0000 7f9a5da96700 1 -- 172.21.15.97:0/3557681534 wait complete. 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 Processor -- start 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 -- start start 2023-10-08T09:54:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a581008a0 0x7f9a5819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 0x7f9a5819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5819e140 con 0x7f9a580ff6b0 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5819e2b0 con 0x7f9a58101240 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a5da96700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5819e470 con 0x7f9a581008a0 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a577fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a577fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55366/0 (socket says 172.21.15.97:55366) 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.305+0000 7f9a577fe700 1 -- 172.21.15.97:0/974381018 learned_addr learned my addr 172.21.15.97:0/974381018 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a57fff700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 0x7f9a5819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a56ffd700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a581008a0 0x7f9a5819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 -- 172.21.15.97:0/974381018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a581008a0 msgr2=0x7f9a5819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a581008a0 0x7f9a5819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 -- 172.21.15.97:0/974381018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 msgr2=0x7f9a5819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 0x7f9a5819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 -- 172.21.15.97:0/974381018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a480095f0 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a577fe700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c990 tx=0x7f9a4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c007b00 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5819e750 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4c007c60 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c018660 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.306+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5819ecb0 con 0x7f9a580ff6b0 2023-10-08T09:54:10.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.307+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a58106a60 con 0x7f9a580ff6b0 2023-10-08T09:54:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.308+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9a4c00f450 con 0x7f9a580ff6b0 2023-10-08T09:54:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.308+0000 7f9a54ff9700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 0x7f9a400a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.309+0000 7f9a56ffd700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 0x7f9a400a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.309+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9a4c0ccf80 con 0x7f9a580ff6b0 2023-10-08T09:54:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.309+0000 7f9a56ffd700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 0x7f9a400a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a48003820 tx=0x7f9a48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.312+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9a4c096500 con 0x7f9a580ff6b0 2023-10-08T09:54:10.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:10 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3760302411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:10 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3214970703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:10 smithi097 ceph-mon[99491]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 118 B/s wr, 0 op/s 2023-10-08T09:54:10.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:10 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1320135640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.504+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a5810b170 con 0x7f9a580ff6b0 2023-10-08T09:54:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.505+0000 7f9a54ff9700 1 -- 172.21.15.97:0/974381018 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a4c099b20 con 0x7f9a580ff6b0 2023-10-08T09:54:10.506 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:54:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 msgr2=0x7f9a400a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 0x7f9a400a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a48003820 tx=0x7f9a48005fd0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 msgr2=0x7f9a58069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c990 tx=0x7f9a4c00cd50 comp rx=0 tx=0).stop 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 shutdown_connections 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9a400a1500 0x7f9a400a39c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a58101240 0x7f9a5819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a580ff6b0 0x7f9a58069550 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 --2- 172.21.15.97:0/974381018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a581008a0 0x7f9a5819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 >> 172.21.15.97:0/974381018 conn(0x7f9a580fac40 msgr2=0x7f9a58103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 shutdown_connections 2023-10-08T09:54:10.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.507+0000 7f9a5da96700 1 -- 172.21.15.97:0/974381018 wait complete. 2023-10-08T09:54:10.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T09:54:10.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:10 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3760302411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:10 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3214970703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:10 smithi186 ceph-mon[104393]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 118 B/s wr, 0 op/s 2023-10-08T09:54:10.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:10 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1320135640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 -- 172.21.15.97:0/1103213759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c0ff6b0 msgr2=0x7f749c0ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 --2- 172.21.15.97:0/1103213759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c0ff6b0 0x7f749c0ffad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f74840099c0 tx=0x7f7484009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 -- 172.21.15.97:0/1103213759 shutdown_connections 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 --2- 172.21.15.97:0/1103213759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c101240 0x7f749c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 --2- 172.21.15.97:0/1103213759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c0ff6b0 0x7f749c0ffad0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 --2- 172.21.15.97:0/1103213759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c1008a0 0x7f749c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 -- 172.21.15.97:0/1103213759 >> 172.21.15.97:0/1103213759 conn(0x7f749c0fac00 msgr2=0x7f749c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 -- 172.21.15.97:0/1103213759 shutdown_connections 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.641+0000 7f74a1495700 1 -- 172.21.15.97:0/1103213759 wait complete. 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 Processor -- start 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 -- start start 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c0ff6b0 0x7f749c195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 0x7f749c195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19d1b0 con 0x7f749c101240 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19d2f0 con 0x7f749c1008a0 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f74a1495700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c19d480 con 0x7f749c0ff6b0 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55382/0 (socket says 172.21.15.97:55382) 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 -- 172.21.15.97:0/1271289209 learned_addr learned my addr 172.21.15.97:0/1271289209 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749affd700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c0ff6b0 0x7f749c195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749a7fc700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 0x7f749c195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 -- 172.21.15.97:0/1271289209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c0ff6b0 msgr2=0x7f749c195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c0ff6b0 0x7f749c195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 -- 172.21.15.97:0/1271289209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 msgr2=0x7f749c195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 0x7f749c195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749b7fe700 1 -- 172.21.15.97:0/1271289209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840095f0 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.642+0000 7f749a7fc700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 0x7f749c195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f749b7fe700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f749400b970 tx=0x7f749400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400c9b0 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7494014440 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c19d760 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940075d0 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.643+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c19dc20 con 0x7f749c101240 2023-10-08T09:54:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.644+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c106b50 con 0x7f749c101240 2023-10-08T09:54:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.645+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f74940145b0 con 0x7f749c101240 2023-10-08T09:54:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.645+0000 7f7493fff700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 0x7f74880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.645+0000 7f749affd700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 0x7f74880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.645+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7494006e40 con 0x7f749c101240 2023-10-08T09:54:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.646+0000 7f749affd700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 0x7f74880a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7484009990 tx=0x7f7484005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:10.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.649+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f749404f100 con 0x7f749c101240 2023-10-08T09:54:10.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:10 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3760302411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:10 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3214970703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:10 smithi178 ceph-mon[103156]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 118 B/s wr, 0 op/s 2023-10-08T09:54:10.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:10 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1320135640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.856+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f749c10b180 con 0x7f749c101240 2023-10-08T09:54:10.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.857+0000 7f7493fff700 1 -- 172.21.15.97:0/1271289209 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+591 (secure 0 0 0) 0x7f7494095b10 con 0x7f749c101240 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4256028615 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/1217300114 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4047632094 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/353854862 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2928584164 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3484524433 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 msgr2=0x7f74880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 0x7f74880a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7484009990 tx=0x7f7484005e80 comp rx=0 tx=0).stop 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 msgr2=0x7f749c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f749400b970 tx=0x7f749400bd30 comp rx=0 tx=0).stop 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 shutdown_connections 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f74880a1480 0x7f74880a3940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c101240 0x7f749c19cbe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f749c0ff6b0 0x7f749c195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 --2- 172.21.15.97:0/1271289209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749c1008a0 0x7f749c195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:10.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 >> 172.21.15.97:0/1271289209 conn(0x7f749c0fac00 msgr2=0x7f749c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:10.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 shutdown_connections 2023-10-08T09:54:10.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:10.859+0000 7f74a1495700 1 -- 172.21.15.97:0/1271289209 wait complete. 2023-10-08T09:54:10.861 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2023-10-08T09:54:10.872 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:54:10.872 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:54:10.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 -- 172.21.15.97:0/3265291721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2480ff5d0 msgr2=0x7fd2480ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/3265291721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2480ff5d0 0x7fd2480ff9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2300099c0 tx=0x7fd230009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 -- 172.21.15.97:0/3265291721 shutdown_connections 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/3265291721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 0x7fd248107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/3265291721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2480ff5d0 0x7fd2480ff9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/3265291721 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2481007c0 0x7fd248100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 -- 172.21.15.97:0/3265291721 >> 172.21.15.97:0/3265291721 conn(0x7fd2480fab40 msgr2=0x7fd2480fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 -- 172.21.15.97:0/3265291721 shutdown_connections 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 -- 172.21.15.97:0/3265291721 wait complete. 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.051+0000 7fd24d6fe700 1 Processor -- start 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 -- start start 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2480ff5d0 0x7fd248069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 0x7fd24819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481a02f0 con 0x7fd2481007c0 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481a0460 con 0x7fd248101160 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd24d6fe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481a0620 con 0x7fd2480ff5d0 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55394/0 (socket says 172.21.15.97:55394) 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 -- 172.21.15.97:0/1374740816 learned_addr learned my addr 172.21.15.97:0/1374740816 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2477fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 0x7fd24819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd246ffd700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2480ff5d0 0x7fd248069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 -- 172.21.15.97:0/1374740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2480ff5d0 msgr2=0x7fd248069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2480ff5d0 0x7fd248069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 -- 172.21.15.97:0/1374740816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 msgr2=0x7fd24819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 0x7fd24819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 -- 172.21.15.97:0/1374740816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2300095f0 con 0x7fd2481007c0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.052+0000 7fd2467fc700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c920 tx=0x7fd23800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.053+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238004a30 con 0x7fd2481007c0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.053+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2380054e0 con 0x7fd2481007c0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.053+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2481a0900 con 0x7fd2481007c0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.053+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238007bd0 con 0x7fd2481007c0 2023-10-08T09:54:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.053+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2481a0e10 con 0x7fd2481007c0 2023-10-08T09:54:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.054+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2481069a0 con 0x7fd2481007c0 2023-10-08T09:54:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.055+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd238005000 con 0x7fd2481007c0 2023-10-08T09:54:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.055+0000 7fd23ffff700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 0x7fd2340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.055+0000 7fd246ffd700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 0x7fd2340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.055+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd2380cca20 con 0x7fd2481007c0 2023-10-08T09:54:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.056+0000 7fd246ffd700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 0x7fd2340a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd230003820 tx=0x7fd230005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.059+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd238095e90 con 0x7fd2481007c0 2023-10-08T09:54:11.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.255+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2480678a0 con 0x7fd2481007c0 2023-10-08T09:54:11.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.256+0000 7fd23ffff700 1 -- 172.21.15.97:0/1374740816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7fd23801d090 con 0x7fd2481007c0 2023-10-08T09:54:11.257 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:11.257 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:11.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 msgr2=0x7fd2340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 0x7fd2340a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd230003820 tx=0x7fd230005920 comp rx=0 tx=0).stop 2023-10-08T09:54:11.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 msgr2=0x7fd24819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c920 tx=0x7fd23800cc30 comp rx=0 tx=0).stop 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 shutdown_connections 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd2340a1480 0x7fd2340a3940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd248101160 0x7fd24819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2480ff5d0 0x7fd248069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 --2- 172.21.15.97:0/1374740816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2481007c0 0x7fd24819d740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 >> 172.21.15.97:0/1374740816 conn(0x7fd2480fab40 msgr2=0x7fd248103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 shutdown_connections 2023-10-08T09:54:11.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.258+0000 7fd24d6fe700 1 -- 172.21.15.97:0/1374740816 wait complete. 2023-10-08T09:54:11.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:54:11.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:54:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 -- 172.21.15.97:0/752907906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c103bf0 msgr2=0x7faf0c105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 --2- 172.21.15.97:0/752907906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c103bf0 0x7faf0c105fe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf08009990 tx=0x7faf08009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 -- 172.21.15.97:0/752907906 shutdown_connections 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 --2- 172.21.15.97:0/752907906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c106520 0x7faf0c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 --2- 172.21.15.97:0/752907906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c1012c0 0x7faf0c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 --2- 172.21.15.97:0/752907906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c103bf0 0x7faf0c105fe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 -- 172.21.15.97:0/752907906 >> 172.21.15.97:0/752907906 conn(0x7faf0c0fac10 msgr2=0x7faf0c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 -- 172.21.15.97:0/752907906 shutdown_connections 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.394+0000 7faf1437a700 1 -- 172.21.15.97:0/752907906 wait complete. 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 Processor -- start 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 -- start start 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 0x7faf0c0ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 0x7faf0c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0fff50 con 0x7faf0c1012c0 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1000c0 con 0x7faf0c106520 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf1437a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1a0820 con 0x7faf0c103bf0 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55406/0 (socket says 172.21.15.97:55406) 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 -- 172.21.15.97:0/2974494957 learned_addr learned my addr 172.21.15.97:0/2974494957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12917700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 0x7faf0c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf11915700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 0x7faf0c0ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 -- 172.21.15.97:0/2974494957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 msgr2=0x7faf0c0ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 0x7faf0c0ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 -- 172.21.15.97:0/2974494957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 msgr2=0x7faf0c0ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 0x7faf0c0ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12116700 1 -- 172.21.15.97:0/2974494957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf080095f0 con 0x7faf0c1012c0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf12917700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 0x7faf0c0ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.395+0000 7faf11915700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 0x7faf0c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf12116700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faefc00b530 tx=0x7faefc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc011830 con 0x7faf0c1012c0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc011e50 con 0x7faf0c1012c0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c1a0ab0 con 0x7faf0c1012c0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc010e50 con 0x7faf0c1012c0 2023-10-08T09:54:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.396+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1a1010 con 0x7faf0c1012c0 2023-10-08T09:54:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.397+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c0fc790 con 0x7faf0c1012c0 2023-10-08T09:54:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.398+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faefc0104f0 con 0x7faf0c1012c0 2023-10-08T09:54:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.398+0000 7faf037fe700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 0x7faef80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.398+0000 7faf11915700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 0x7faef80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.398+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faefc0cca90 con 0x7faf0c1012c0 2023-10-08T09:54:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.399+0000 7faf11915700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 0x7faef80a39c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf080037f0 tx=0x7faf08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.403+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faefc095ea0 con 0x7faf0c1012c0 2023-10-08T09:54:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:11 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/974381018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:11 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1271289209' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:11 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1374740816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:11.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.593+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf0c0fc170 con 0x7faf0c1012c0 2023-10-08T09:54:11.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.594+0000 7faf037fe700 1 -- 172.21.15.97:0/2974494957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7faefc0150a0 con 0x7faf0c1012c0 2023-10-08T09:54:11.595 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 msgr2=0x7faef80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 0x7faef80a39c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf080037f0 tx=0x7faf08005fb0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 msgr2=0x7faf0c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faefc00b530 tx=0x7faefc00b8f0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 shutdown_connections 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faef80a1500 0x7faef80a39c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf0c106520 0x7faf0c0ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c1012c0 0x7faf0c100be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 --2- 172.21.15.97:0/2974494957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c103bf0 0x7faf0c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 >> 172.21.15.97:0/2974494957 conn(0x7faf0c0fac10 msgr2=0x7faf0c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 shutdown_connections 2023-10-08T09:54:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.596+0000 7faf1437a700 1 -- 172.21.15.97:0/2974494957 wait complete. 2023-10-08T09:54:11.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:11 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/974381018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:11.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:11 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1271289209' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:11.607 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:11 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1374740816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:11.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:54:11.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:11 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/974381018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:11.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:11 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1271289209' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:11.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:11 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1374740816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 -- 172.21.15.97:0/810173464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d70101230 msgr2=0x7f4d70107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 --2- 172.21.15.97:0/810173464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d70101230 0x7f4d70107ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800b0d0 tx=0x7f4d6800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 -- 172.21.15.97:0/810173464 shutdown_connections 2023-10-08T09:54:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 --2- 172.21.15.97:0/810173464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d70101230 0x7f4d70107ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 --2- 172.21.15.97:0/810173464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d700ff6a0 0x7f4d700ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 --2- 172.21.15.97:0/810173464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70100890 0x7f4d70100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 -- 172.21.15.97:0/810173464 >> 172.21.15.97:0/810173464 conn(0x7f4d700fac10 msgr2=0x7f4d700fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 -- 172.21.15.97:0/810173464 shutdown_connections 2023-10-08T09:54:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.733+0000 7f4d748aa700 1 -- 172.21.15.97:0/810173464 wait complete. 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 Processor -- start 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 -- start start 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d70100890 0x7f4d70195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70101230 0x7f4d7019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019d220 con 0x7f4d700ff6a0 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019d360 con 0x7f4d70100890 2023-10-08T09:54:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d748aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019d4f0 con 0x7f4d70101230 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55420/0 (socket says 172.21.15.97:55420) 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.734+0000 7f4d6e59c700 1 -- 172.21.15.97:0/3744208470 learned_addr learned my addr 172.21.15.97:0/3744208470 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6ed9d700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70101230 0x7f4d7019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6dd9b700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d70100890 0x7f4d70195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 -- 172.21.15.97:0/3744208470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70101230 msgr2=0x7f4d7019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70101230 0x7f4d7019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 -- 172.21.15.97:0/3744208470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d70100890 msgr2=0x7f4d70195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d70100890 0x7f4d70195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 -- 172.21.15.97:0/3744208470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68009d00 con 0x7f4d700ff6a0 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d6e59c700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d640098f0 tx=0x7f4d64009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64012070 con 0x7f4d700ff6a0 2023-10-08T09:54:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d640044e0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70067890 con 0x7f4d700ff6a0 2023-10-08T09:54:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64007d10 con 0x7f4d700ff6a0 2023-10-08T09:54:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.735+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d70067da0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.736+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70067530 con 0x7f4d700ff6a0 2023-10-08T09:54:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.737+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4d64016030 con 0x7f4d700ff6a0 2023-10-08T09:54:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.737+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 0x7f4d580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.738+0000 7f4d6dd9b700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 0x7f4d580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.738+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4d640ce1b0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.738+0000 7f4d6dd9b700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 0x7f4d580a39c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d60005950 tx=0x7f4d600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:11.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.742+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4d640975c0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.932+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d701064d0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.932+0000 7f4d5f7fe700 1 -- 172.21.15.97:0/3744208470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d6409abe0 con 0x7f4d700ff6a0 2023-10-08T09:54:11.934 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:54:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 msgr2=0x7f4d580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 0x7f4d580a39c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d60005950 tx=0x7f4d600058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 msgr2=0x7f4d701956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d640098f0 tx=0x7f4d64009cb0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 shutdown_connections 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4d580a1500 0x7f4d580a39c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d70101230 0x7f4d7019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d700ff6a0 0x7f4d701956f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 --2- 172.21.15.97:0/3744208470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d70100890 0x7f4d70195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.934+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 >> 172.21.15.97:0/3744208470 conn(0x7f4d700fac10 msgr2=0x7f4d70103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.935+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 shutdown_connections 2023-10-08T09:54:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:11.935+0000 7f4d748aa700 1 -- 172.21.15.97:0/3744208470 wait complete. 2023-10-08T09:54:11.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:54:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/239675505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fff70 msgr2=0x7fcd581003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/239675505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fff70 0x7fcd581003f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009990 tx=0x7fcd48009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/239675505 shutdown_connections 2023-10-08T09:54:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/239675505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd58100930 0x7fcd58107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/239675505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd580fee20 0x7fcd580ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/239675505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fff70 0x7fcd581003f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/239675505 >> 172.21.15.97:0/239675505 conn(0x7fcd580fa390 msgr2=0x7fcd580fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/239675505 shutdown_connections 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.070+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/239675505 wait complete. 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 Processor -- start 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 -- start start 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd580fff70 0x7fcd58195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd58100930 0x7fcd5819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819d140 con 0x7fcd580fee20 2023-10-08T09:54:12.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819d280 con 0x7fcd580fff70 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5e4c0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819d410 con 0x7fcd58100930 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55434/0 (socket says 172.21.15.97:55434) 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 -- 172.21.15.97:0/3054101809 learned_addr learned my addr 172.21.15.97:0/3054101809 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd577fe700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd580fff70 0x7fcd58195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd5ca5d700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd58100930 0x7fcd5819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 -- 172.21.15.97:0/3054101809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd58100930 msgr2=0x7fcd5819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd58100930 0x7fcd5819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 -- 172.21.15.97:0/3054101809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd580fff70 msgr2=0x7fcd58195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd580fff70 0x7fcd58195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 -- 172.21.15.97:0/3054101809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480095f0 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.071+0000 7fcd57fff700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d980 tx=0x7fcd4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.072+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004a30 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.072+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5819d6f0 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.072+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4c0054e0 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.072+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c009c90 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.072+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5819dbb0 con 0x7fcd580fee20 2023-10-08T09:54:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.073+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58106260 con 0x7fcd580fee20 2023-10-08T09:54:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.074+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcd4c004c20 con 0x7fcd580fee20 2023-10-08T09:54:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.075+0000 7fcd557fa700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 0x7fcd400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.075+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcd4c0cd1b0 con 0x7fcd580fee20 2023-10-08T09:54:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.075+0000 7fcd577fe700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 0x7fcd400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.075+0000 7fcd577fe700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 0x7fcd400a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800b2e0 tx=0x7fcd4800b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.078+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcd4c096620 con 0x7fcd580fee20 2023-10-08T09:54:12.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.271+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd5810aef0 con 0x7fcd580fee20 2023-10-08T09:54:12.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.274+0000 7fcd557fa700 1 -- 172.21.15.97:0/3054101809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd4c017020 con 0x7fcd580fee20 2023-10-08T09:54:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 msgr2=0x7fcd400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 0x7fcd400a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800b2e0 tx=0x7fcd4800b6f0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 msgr2=0x7fcd58195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d980 tx=0x7fcd4c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 shutdown_connections 2023-10-08T09:54:12.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcd400a1480 0x7fcd400a3940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd58100930 0x7fcd5819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd580fee20 0x7fcd58195610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 --2- 172.21.15.97:0/3054101809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd580fff70 0x7fcd58195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 >> 172.21.15.97:0/3054101809 conn(0x7fcd580fa390 msgr2=0x7fcd581033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 shutdown_connections 2023-10-08T09:54:12.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.276+0000 7fcd5e4c0700 1 -- 172.21.15.97:0/3054101809 wait complete. 2023-10-08T09:54:12.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/498956395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c0100890 msgr2=0x7fb8c0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/498956395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c0100890 0x7fb8c0100cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00099c0 tx=0x7fb8b0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/498956395 shutdown_connections 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/498956395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0101230 0x7fb8c0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/498956395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/498956395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c0100890 0x7fb8c0100cf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/498956395 >> 172.21.15.97:0/498956395 conn(0x7fb8c00fac10 msgr2=0x7fb8c00fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/498956395 shutdown_connections 2023-10-08T09:54:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.413+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/498956395 wait complete. 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 Processor -- start 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 -- start start 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 0x7fb8c0195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 0x7fb8c019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019d150 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019d290 con 0x7fb8c0101230 2023-10-08T09:54:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8c4ecf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c019d420 con 0x7fb8c0100890 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55448/0 (socket says 172.21.15.97:55448) 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8be59c700 1 -- 172.21.15.97:0/2204977707 learned_addr learned my addr 172.21.15.97:0/2204977707 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8bdd9b700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 0x7fb8c0195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8bed9d700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 0x7fb8c019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8be59c700 1 -- 172.21.15.97:0/2204977707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 msgr2=0x7fb8c0195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.414+0000 7fb8be59c700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 0x7fb8c0195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8be59c700 1 -- 172.21.15.97:0/2204977707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 msgr2=0x7fb8c019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8be59c700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 0x7fb8c019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8be59c700 1 -- 172.21.15.97:0/2204977707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00095f0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8bdd9b700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 0x7fb8c0195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8bed9d700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 0x7fb8c019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8be59c700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4007b50 tx=0x7fb8b4007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4011070 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c0067890 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8b400c2e0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400e5c0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.415+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c0067df0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.416+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c010b7f0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.417+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb8b4003f20 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.417+0000 7fb8af7fe700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 0x7fb8a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.417+0000 7fb8bdd9b700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 0x7fb8a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.417+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb8b40ce2e0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.418+0000 7fb8bdd9b700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 0x7fb8a80a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0000c00 tx=0x7fb8b00037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.422+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb8b40977e0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.613+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb8c010b0e0 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.613+0000 7fb8af7fe700 1 -- 172.21.15.97:0/2204977707 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb8b409ae00 con 0x7fb8c00ff6a0 2023-10-08T09:54:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:54:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.615+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 msgr2=0x7fb8a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.615+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 0x7fb8a80a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0000c00 tx=0x7fb8b00037c0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.615+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 msgr2=0x7fb8c0195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.615+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4007b50 tx=0x7fb8b4007f10 comp rx=0 tx=0).stop 2023-10-08T09:54:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 shutdown_connections 2023-10-08T09:54:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb8a80a1480 0x7fb8a80a3940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c0101230 0x7fb8c019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c00ff6a0 0x7fb8c0195620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 --2- 172.21.15.97:0/2204977707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8c0100890 0x7fb8c0195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 >> 172.21.15.97:0/2204977707 conn(0x7fb8c00fac10 msgr2=0x7fb8c0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 shutdown_connections 2023-10-08T09:54:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.616+0000 7fb8c4ecf700 1 -- 172.21.15.97:0/2204977707 wait complete. 2023-10-08T09:54:12.627 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:12 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2974494957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.628 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:12 smithi097 ceph-mon[99491]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2023-10-08T09:54:12.628 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:12 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3744208470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T09:54:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:12 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2974494957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:12 smithi178 ceph-mon[103156]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2023-10-08T09:54:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:12 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3744208470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 -- 172.21.15.97:0/427876838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8814101240 msgr2=0x7f8814107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 --2- 172.21.15.97:0/427876838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8814101240 0x7f8814107af0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b0d0 tx=0x7f880c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 -- 172.21.15.97:0/427876838 shutdown_connections 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 --2- 172.21.15.97:0/427876838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8814101240 0x7f8814107af0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 --2- 172.21.15.97:0/427876838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88140ff6b0 0x7f88140ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 --2- 172.21.15.97:0/427876838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 0x7f8814100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 -- 172.21.15.97:0/427876838 >> 172.21.15.97:0/427876838 conn(0x7f88140fac40 msgr2=0x7f88140fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.750+0000 7f881acba700 1 -- 172.21.15.97:0/427876838 shutdown_connections 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 -- 172.21.15.97:0/427876838 wait complete. 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 Processor -- start 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 -- start start 2023-10-08T09:54:12.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 0x7f8814195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814101240 0x7f881419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419d240 con 0x7f88140ff6b0 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419d380 con 0x7f8814101240 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f881acba700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419d510 con 0x7f88141008a0 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f8818a56700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f8818a56700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55462/0 (socket says 172.21.15.97:55462) 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f8818a56700 1 -- 172.21.15.97:0/3008990330 learned_addr learned my addr 172.21.15.97:0/3008990330 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.751+0000 7f8813fff700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 0x7f8814195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 -- 172.21.15.97:0/3008990330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 msgr2=0x7f8814195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8819257700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814101240 0x7f881419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 0x7f8814195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 -- 172.21.15.97:0/3008990330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814101240 msgr2=0x7f881419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814101240 0x7f881419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 -- 172.21.15.97:0/3008990330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c009d00 con 0x7f88140ff6b0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8818a56700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880800c970 tx=0x7f880800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808004220 con 0x7f88140ff6b0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8808007d60 con 0x7f88140ff6b0 2023-10-08T09:54:12.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881419d7f0 con 0x7f88140ff6b0 2023-10-08T09:54:12.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808003c90 con 0x7f88140ff6b0 2023-10-08T09:54:12.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.752+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881419dd00 con 0x7f88140ff6b0 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.753+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814106a60 con 0x7f88140ff6b0 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.754+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8808004380 con 0x7f88140ff6b0 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.754+0000 7f8811ffb700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 0x7f88040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.755+0000 7f8813fff700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 0x7f88040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.755+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f88080ccb10 con 0x7f88140ff6b0 2023-10-08T09:54:12.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.755+0000 7f8813fff700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 0x7f88040a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8800009c60 tx=0x7f8800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:12.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.759+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8808095f20 con 0x7f88140ff6b0 2023-10-08T09:54:12.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:12 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2974494957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:12 smithi186 ceph-mon[104393]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 0 op/s 2023-10-08T09:54:12.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:12 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3744208470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:54:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.967+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f881410b180 con 0x7f88140ff6b0 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.968+0000 7f8811ffb700 1 -- 172.21.15.97:0/3008990330 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+591 (secure 0 0 0) 0x7f880801d260 con 0x7f88140ff6b0 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4256028615 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/1217300114 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4047632094 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:12.969 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/353854862 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2928584164 2023-10-09T09:47:22.724740+0000 2023-10-08T09:54:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3484524433 2023-10-09T09:47:45.797606+0000 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 msgr2=0x7f88040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 0x7f88040a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8800009c60 tx=0x7f8800009260 comp rx=0 tx=0).stop 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 msgr2=0x7f8814195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880800c970 tx=0x7f880800cc80 comp rx=0 tx=0).stop 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 shutdown_connections 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f88040a1480 0x7f88040a3940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8814101240 0x7f881419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88140ff6b0 0x7f8814195710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 --2- 172.21.15.97:0/3008990330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88141008a0 0x7f8814195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 >> 172.21.15.97:0/3008990330 conn(0x7f88140fac40 msgr2=0x7f8814103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 shutdown_connections 2023-10-08T09:54:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:12.970+0000 7f881acba700 1 -- 172.21.15.97:0/3008990330 wait complete. 2023-10-08T09:54:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2023-10-08T09:54:12.982 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-10-08T09:54:12.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:12.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:54:13.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[158444]: fuse finished with error 0 and tester_r 0 2023-10-08T09:54:13.022 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T09:54:13.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:13 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3054101809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:13 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2204977707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:13 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3008990330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:13 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3054101809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:13 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2204977707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:13 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3008990330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:13 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3054101809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:13 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2204977707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:54:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:13 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3008990330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:54:14.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:14 smithi097 ceph-mon[99491]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 263 B/s wr, 0 op/s 2023-10-08T09:54:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:14 smithi178 ceph-mon[103156]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 263 B/s wr, 0 op/s 2023-10-08T09:54:14.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:14 smithi186 ceph-mon[104393]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 263 B/s wr, 0 op/s 2023-10-08T09:54:17.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:17 smithi097 ceph-mon[99491]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:17 smithi178 ceph-mon[103156]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:17 smithi186 ceph-mon[104393]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:19.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:54:19.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:54:19.123 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-08T09:54:19.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/3025491136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24100790 msgr2=0x7f2e24100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/3025491136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24100790 0x7f2e24100bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e140099c0 tx=0x7f2e14009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/3025491136 shutdown_connections 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/3025491136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e24101130 0x7f2e241079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/3025491136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e240ff5a0 0x7f2e240ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/3025491136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24100790 0x7f2e24100bf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/3025491136 >> 172.21.15.97:0/3025491136 conn(0x7f2e240fab10 msgr2=0x7f2e240fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/3025491136 shutdown_connections 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/3025491136 wait complete. 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 Processor -- start 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.331+0000 7f2e2a8fe700 1 -- start start 2023-10-08T09:54:19.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 0x7f2e24196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e24100790 0x7f2e24196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419e060 con 0x7f2e24101130 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419e1a0 con 0x7f2e240ff5a0 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419e330 con 0x7f2e24100790 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55480/0 (socket says 172.21.15.97:55480) 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 -- 172.21.15.97:0/2226989037 learned_addr learned my addr 172.21.15.97:0/2226989037 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e23fff700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 0x7f2e24196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 -- 172.21.15.97:0/2226989037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e24100790 msgr2=0x7f2e24196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e24100790 0x7f2e24196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 -- 172.21.15.97:0/2226989037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 msgr2=0x7f2e24196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 0x7f2e24196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 -- 172.21.15.97:0/2226989037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140095f0 con 0x7f2e24101130 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e23fff700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 0x7f2e24196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e28e9b700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c007d70 tx=0x7f2e1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00ad40 con 0x7f2e24101130 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e1c0128e0 con 0x7f2e24101130 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2419e610 con 0x7f2e24101130 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011780 con 0x7f2e24101130 2023-10-08T09:54:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.332+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2419ead0 con 0x7f2e24101130 2023-10-08T09:54:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.333+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24067450 con 0x7f2e24101130 2023-10-08T09:54:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.334+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2e1c012a50 con 0x7f2e24101130 2023-10-08T09:54:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.335+0000 7f2e217fa700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 0x7f2e0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.335+0000 7f2e23fff700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 0x7f2e0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.335+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2e1c0cd6a0 con 0x7f2e24101130 2023-10-08T09:54:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.336+0000 7f2e23fff700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 0x7f2e0c0a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005fd0 tx=0x7f2e18005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:19.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.339+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e1c096b10 con 0x7f2e24101130 2023-10-08T09:54:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:19 smithi097 ceph-mon[99491]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:19 smithi178 ceph-mon[103156]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:19 smithi186 ceph-mon[104393]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.532+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e2410adc0 con 0x7f2e24101130 2023-10-08T09:54:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.533+0000 7f2e217fa700 1 -- 172.21.15.97:0/2226989037 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f2e1c09a130 con 0x7f2e24101130 2023-10-08T09:54:19.535 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:19.535 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 msgr2=0x7f2e0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 0x7f2e0c0a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005fd0 tx=0x7f2e18005e40 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 msgr2=0x7f2e2419da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c007d70 tx=0x7f2e1c00a340 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 shutdown_connections 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2e0c0a1480 0x7f2e0c0a3940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e24101130 0x7f2e2419da90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e240ff5a0 0x7f2e24196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 --2- 172.21.15.97:0/2226989037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e24100790 0x7f2e24196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 >> 172.21.15.97:0/2226989037 conn(0x7f2e240fab10 msgr2=0x7f2e240fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 shutdown_connections 2023-10-08T09:54:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.536+0000 7f2e2a8fe700 1 -- 172.21.15.97:0/2226989037 wait complete. 2023-10-08T09:54:19.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:19.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/658894879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24101130 msgr2=0x7f1d241079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/658894879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24101130 0x7f1d241079e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00b0d0 tx=0x7f1d1c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/658894879 shutdown_connections 2023-10-08T09:54:19.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/658894879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24101130 0x7f1d241079e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/658894879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d240ff5a0 0x7f1d240ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/658894879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d24100790 0x7f1d24100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/658894879 >> 172.21.15.97:0/658894879 conn(0x7f1d240fab10 msgr2=0x7f1d240fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/658894879 shutdown_connections 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/658894879 wait complete. 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 Processor -- start 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.670+0000 7f1d2ac2a700 1 -- start start 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d24100790 0x7f1d24195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 0x7f1d2419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2419d110 con 0x7f1d240ff5a0 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2419d250 con 0x7f1d24100790 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2419d3e0 con 0x7f1d24101130 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55490/0 (socket says 172.21.15.97:55490) 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 -- 172.21.15.97:0/3353136433 learned_addr learned my addr 172.21.15.97:0/3353136433 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d291c7700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 0x7f1d2419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 -- 172.21.15.97:0/3353136433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 msgr2=0x7f1d2419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 0x7f1d2419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 -- 172.21.15.97:0/3353136433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d24100790 msgr2=0x7f1d24195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d24100790 0x7f1d24195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 -- 172.21.15.97:0/3353136433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c009d00 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d291c7700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 0x7f1d2419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d289c6700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d980 tx=0x7f1d1800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800de60 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2419d6c0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d180044e0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.671+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18003be0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.672+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2419dbd0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.672+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d24106950 con 0x7f1d240ff5a0 2023-10-08T09:54:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.673+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1d18009bb0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.674+0000 7f1d21ffb700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 0x7f1d140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.674+0000 7f1d23fff700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 0x7f1d140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:19.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.674+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d180ccc80 con 0x7f1d240ff5a0 2023-10-08T09:54:19.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.674+0000 7f1d23fff700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 0x7f1d140a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000a990 tx=0x7f1d10005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.678+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d180960f0 con 0x7f1d240ff5a0 2023-10-08T09:54:19.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.871+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d24067450 con 0x7f1d240ff5a0 2023-10-08T09:54:19.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.873+0000 7f1d21ffb700 1 -- 172.21.15.97:0/3353136433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f1d18009940 con 0x7f1d240ff5a0 2023-10-08T09:54:19.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:19.874 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:19.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 msgr2=0x7f1d140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 0x7f1d140a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000a990 tx=0x7f1d10005c80 comp rx=0 tx=0).stop 2023-10-08T09:54:19.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 msgr2=0x7f1d241955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:19.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d980 tx=0x7f1d1800dd40 comp rx=0 tx=0).stop 2023-10-08T09:54:19.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 shutdown_connections 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1d140a1480 0x7f1d140a3940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d24101130 0x7f1d2419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d240ff5a0 0x7f1d241955e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 --2- 172.21.15.97:0/3353136433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d24100790 0x7f1d24195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 >> 172.21.15.97:0/3353136433 conn(0x7f1d240fab10 msgr2=0x7f1d24103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 shutdown_connections 2023-10-08T09:54:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:19.875+0000 7f1d2ac2a700 1 -- 172.21.15.97:0/3353136433 wait complete. 2023-10-08T09:54:19.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/1805561377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 msgr2=0x7fe9d00ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/1805561377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d00ffac0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40099c0 tx=0x7fe9c4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/1805561377 shutdown_connections 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/1805561377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0101230 0x7fe9d0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/1805561377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d00ffac0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/1805561377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0100890 0x7fe9d0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/1805561377 >> 172.21.15.97:0/1805561377 conn(0x7fe9d00fac10 msgr2=0x7fe9d00fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/1805561377 shutdown_connections 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/1805561377 wait complete. 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.012+0000 7fe9d5ee2700 1 Processor -- start 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 -- start start 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0100890 0x7fe9d019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 0x7fe9d019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019e2b0 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019e420 con 0x7fe9d0101230 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d019e5e0 con 0x7fe9d0100890 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55504/0 (socket says 172.21.15.97:55504) 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 -- 172.21.15.97:0/3649292657 learned_addr learned my addr 172.21.15.97:0/3649292657 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9ceffd700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0100890 0x7fe9d019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cffff700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 0x7fe9d019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 -- 172.21.15.97:0/3649292657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0100890 msgr2=0x7fe9d019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0100890 0x7fe9d019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 -- 172.21.15.97:0/3649292657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 msgr2=0x7fe9d019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 0x7fe9d019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 -- 172.21.15.97:0/3649292657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40095f0 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cffff700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 0x7fe9d019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9cf7fe700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4003820 tx=0x7fe9c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c401d070 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c40054f0 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.013+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d019e800 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.014+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004a00 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.014+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d019ed00 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.014+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0106b70 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.016+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe9c4003b50 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.016+0000 7fe9ccff9700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 0x7fe9b80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.016+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe9c40ce060 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.016+0000 7fe9ceffd700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 0x7fe9b80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.017+0000 7fe9ceffd700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 0x7fe9b80a39c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0068910 tx=0x7fe9c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.019+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe9c4097560 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:20.267+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe9d0067500 con 0x7fe9d00ff6a0 2023-10-08T09:54:20.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:20 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2226989037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:20.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:20 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3353136433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:20.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:20 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2226989037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:20.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:20 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3353136433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:20.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:20 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2226989037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:20.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:20 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3353136433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:21.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.151+0000 7fe9ccff9700 1 -- 172.21.15.97:0/3649292657 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9c409ab80 con 0x7fe9d00ff6a0 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 msgr2=0x7fe9b80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 0x7fe9b80a39c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0068910 tx=0x7fe9c000b410 comp rx=0 tx=0).stop 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 msgr2=0x7fe9d0068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4003820 tx=0x7fe9c4004730 comp rx=0 tx=0).stop 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 shutdown_connections 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe9b80a1500 0x7fe9b80a39c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9d0101230 0x7fe9d019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9d00ff6a0 0x7fe9d0068370 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 --2- 172.21.15.97:0/3649292657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9d0100890 0x7fe9d019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 >> 172.21.15.97:0/3649292657 conn(0x7fe9d00fac10 msgr2=0x7fe9d0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 shutdown_connections 2023-10-08T09:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.154+0000 7fe9d5ee2700 1 -- 172.21.15.97:0/3649292657 wait complete. 2023-10-08T09:54:21.170 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True} 2023-10-08T09:54:21.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 -- 172.21.15.97:0/691399575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 msgr2=0x7fd0f8100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/691399575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f8100cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40099c0 tx=0x7fd0f4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 -- 172.21.15.97:0/691399575 shutdown_connections 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/691399575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/691399575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 0x7fd0f80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/691399575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f8100cf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 -- 172.21.15.97:0/691399575 >> 172.21.15.97:0/691399575 conn(0x7fd0f80fac10 msgr2=0x7fd0f80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 -- 172.21.15.97:0/691399575 shutdown_connections 2023-10-08T09:54:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.299+0000 7fd0ffc72700 1 -- 172.21.15.97:0/691399575 wait complete. 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 Processor -- start 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 -- start start 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 0x7fd0f8068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819e250 con 0x7fd0f8100890 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819e3c0 con 0x7fd0f80ff6a0 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0ffc72700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819e580 con 0x7fd0f8101230 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55544/0 (socket says 172.21.15.97:55544) 2023-10-08T09:54:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 -- 172.21.15.97:0/414519164 learned_addr learned my addr 172.21.15.97:0/414519164 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fda0e700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 0x7fd0f8068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 -- 172.21.15.97:0/414519164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 msgr2=0x7fd0f819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fe20f700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 -- 172.21.15.97:0/414519164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 msgr2=0x7fd0f8068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 0x7fd0f8068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 -- 172.21.15.97:0/414519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40095f0 con 0x7fd0f8100890 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fe20f700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.300+0000 7fd0fd20d700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40036f0 tx=0x7fd0f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.301+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401d070 con 0x7fd0f8100890 2023-10-08T09:54:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.301+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f819e800 con 0x7fd0f8100890 2023-10-08T09:54:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.301+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f4022470 con 0x7fd0f8100890 2023-10-08T09:54:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.301+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400f6e0 con 0x7fd0f8100890 2023-10-08T09:54:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.301+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f819ecb0 con 0x7fd0f8100890 2023-10-08T09:54:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.302+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8106ac0 con 0x7fd0f8100890 2023-10-08T09:54:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.303+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd0f4022890 con 0x7fd0f8100890 2023-10-08T09:54:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.303+0000 7fd0eeffd700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 0x7fd0e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.303+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0f40ce2a0 con 0x7fd0f8100890 2023-10-08T09:54:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.303+0000 7fd0fda0e700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 0x7fd0e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.304+0000 7fd0fda0e700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 0x7fd0e40a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005fd0 tx=0x7fd0e8005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.308+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd0f405d100 con 0x7fd0f8100890 2023-10-08T09:54:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:21 smithi097 ceph-mon[99491]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:21.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:21 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:54:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:21 smithi178 ceph-mon[103156]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:21 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:54:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:21 smithi186 ceph-mon[104393]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:21 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:54:21.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:21.541+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fd0f810b160 con 0x7fd0f8100890 2023-10-08T09:54:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.161+0000 7fd0eeffd700 1 -- 172.21.15.97:0/414519164 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd0f40977a0 con 0x7fd0f8100890 2023-10-08T09:54:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 msgr2=0x7fd0e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 0x7fd0e40a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005fd0 tx=0x7fd0e8005dd0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 msgr2=0x7fd0f819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40036f0 tx=0x7fd0f4003970 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 shutdown_connections 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd0e40a1480 0x7fd0e40a3940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0f8101230 0x7fd0f819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0f80ff6a0 0x7fd0f8068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 --2- 172.21.15.97:0/414519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f8100890 0x7fd0f819b4c0 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40036f0 tx=0x7fd0f4003970 comp rx=0 tx=0).stop 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.164+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 >> 172.21.15.97:0/414519164 conn(0x7fd0f80fac10 msgr2=0x7fd0f8103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.165+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 shutdown_connections 2023-10-08T09:54:22.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.165+0000 7fd0ffc72700 1 -- 172.21.15.97:0/414519164 wait complete. 2023-10-08T09:54:22.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-08T09:54:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/1155580881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 msgr2=0x7f0b9c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/1155580881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c100d20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0b880099c0 tx=0x7f0b88009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/1155580881 shutdown_connections 2023-10-08T09:54:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/1155580881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c101260 0x7f0b9c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/1155580881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/1155580881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c100d20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/1155580881 >> 172.21.15.97:0/1155580881 conn(0x7f0b9c0fac40 msgr2=0x7f0b9c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/1155580881 shutdown_connections 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.311+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/1155580881 wait complete. 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 Processor -- start 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 -- start start 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c1035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c101260 0x7f0b9c1021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1028e0 con 0x7f0b9c1008c0 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c102a50 con 0x7f0b9c101260 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba2a9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a07e0 con 0x7f0b9c0ff6d0 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55558/0 (socket says 172.21.15.97:55558) 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 -- 172.21.15.97:0/3433325578 learned_addr learned my addr 172.21.15.97:0/3433325578 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba103a700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c101260 0x7f0b9c1021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba0839700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c1035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 -- 172.21.15.97:0/3433325578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 msgr2=0x7f0b9c1035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c1035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 -- 172.21.15.97:0/3433325578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c101260 msgr2=0x7f0b9c1021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c101260 0x7f0b9c1021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 -- 172.21.15.97:0/3433325578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880095f0 con 0x7f0b9c1008c0 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0ba0839700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b9bfff700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b880036f0 tx=0x7f0b88003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.312+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8801d070 con 0x7f0b9c1008c0 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.313+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b88022470 con 0x7f0b9c1008c0 2023-10-08T09:54:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.313+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c1a09e0 con 0x7f0b9c1008c0 2023-10-08T09:54:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.313+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800f6e0 con 0x7f0b9c1008c0 2023-10-08T09:54:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.313+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c1a0ee0 con 0x7f0b9c1008c0 2023-10-08T09:54:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.314+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c067980 con 0x7f0b9c1008c0 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.315+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0b88022890 con 0x7f0b9c1008c0 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.315+0000 7f0b99ffb700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 0x7f0b8c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.315+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0b880ce490 con 0x7f0b9c1008c0 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.315+0000 7f0ba0839700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 0x7f0b8c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.316+0000 7f0ba0839700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 0x7f0b8c0a39c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005fd0 tx=0x7f0b90005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.319+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0b88097990 con 0x7f0b9c1008c0 2023-10-08T09:54:22.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:22 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:54:22.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:22 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:22.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:22 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-08T09:54:22.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:22 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:54:22.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:22 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:22.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:22 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-08T09:54:22.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:22 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3649292657' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:54:22.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:22 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:22.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:22 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-08T09:54:22.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:22.553+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0b9c10aef0 con 0x7f0b9c1008c0 2023-10-08T09:54:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.172+0000 7f0b99ffb700 1 -- 172.21.15.97:0/3433325578 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0b8809afb0 con 0x7f0b9c1008c0 2023-10-08T09:54:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.175+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 msgr2=0x7f0b8c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.175+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 0x7f0b8c0a39c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005fd0 tx=0x7f0b90005dc0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.175+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 msgr2=0x7f0b9c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.175+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0b880036f0 tx=0x7f0b88003970 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 shutdown_connections 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b8c0a1500 0x7f0b8c0a39c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9c101260 0x7f0b9c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b9c0ff6d0 0x7f0b9c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 --2- 172.21.15.97:0/3433325578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9c1008c0 0x7f0b9c101ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 >> 172.21.15.97:0/3433325578 conn(0x7f0b9c0fac40 msgr2=0x7f0b9c068cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 shutdown_connections 2023-10-08T09:54:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.176+0000 7f0ba2a9d700 1 -- 172.21.15.97:0/3433325578 wait complete. 2023-10-08T09:54:23.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 -- 172.21.15.97:0/3176667603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 msgr2=0x7fc8bc101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/3176667603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc101e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00099c0 tx=0x7fc8b0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 -- 172.21.15.97:0/3176667603 shutdown_connections 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/3176667603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/3176667603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 0x7fc8bc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/3176667603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc101e10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 -- 172.21.15.97:0/3176667603 >> 172.21.15.97:0/3176667603 conn(0x7fc8bc0fbd30 msgr2=0x7fc8bc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 -- 172.21.15.97:0/3176667603 shutdown_connections 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 -- 172.21.15.97:0/3176667603 wait complete. 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.324+0000 7fc8c18d2700 1 Processor -- start 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 -- start start 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 0x7fc8bc196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19e2c0 con 0x7fc8bc1019b0 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19e400 con 0x7fc8bc1007c0 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8c18d2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19e590 con 0x7fc8bc102350 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:23.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55572/0 (socket says 172.21.15.97:55572) 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 -- 172.21.15.97:0/295723462 learned_addr learned my addr 172.21.15.97:0/295723462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8baffd700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 0x7fc8bc196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8bb7fe700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 -- 172.21.15.97:0/295723462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 msgr2=0x7fc8bc19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 -- 172.21.15.97:0/295723462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 msgr2=0x7fc8bc196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 0x7fc8bc196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 -- 172.21.15.97:0/295723462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00095f0 con 0x7fc8bc1019b0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8bb7fe700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.325+0000 7fc8ba7fc700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00036f0 tx=0x7fc8b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b001d070 con 0x7fc8bc1019b0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b0004090 con 0x7fc8bc1019b0 2023-10-08T09:54:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc19e810 con 0x7fc8bc1019b0 2023-10-08T09:54:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0022d40 con 0x7fc8bc1019b0 2023-10-08T09:54:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc19ecc0 con 0x7fc8bc1019b0 2023-10-08T09:54:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.326+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc107c10 con 0x7fc8bc1019b0 2023-10-08T09:54:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.328+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc8b0022470 con 0x7fc8bc1019b0 2023-10-08T09:54:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.328+0000 7fc8c08d0700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 0x7fc8a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.328+0000 7fc8baffd700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 0x7fc8a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.328+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc8b00ce4a0 con 0x7fc8bc1019b0 2023-10-08T09:54:23.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.328+0000 7fc8baffd700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 0x7fc8a40a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc101810 tx=0x7fc8ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:23.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.331+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc8b00979a0 con 0x7fc8bc1019b0 2023-10-08T09:54:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:54:23.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:54:23.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:23 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:54:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:54:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-08T09:54:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:54:23.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:23.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:54:23.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:54:23.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:23 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/414519164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:54:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:54:23.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:54:23.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:23 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:54:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:23.567+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc8bc067530 con 0x7fc8bc1019b0 2023-10-08T09:54:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.181+0000 7fc8c08d0700 1 -- 172.21.15.97:0/295723462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc8b009afc0 con 0x7fc8bc1019b0 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 msgr2=0x7fc8a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 0x7fc8a40a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc101810 tx=0x7fc8ac009190 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 msgr2=0x7fc8bc196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00036f0 tx=0x7fc8b00047b0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 shutdown_connections 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc8a40a1480 0x7fc8a40a3940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc102350 0x7fc8bc19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8bc1007c0 0x7fc8bc196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 --2- 172.21.15.97:0/295723462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8bc1019b0 0x7fc8bc196cd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 >> 172.21.15.97:0/295723462 conn(0x7fc8bc0fbd30 msgr2=0x7fc8bc104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 shutdown_connections 2023-10-08T09:54:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.184+0000 7fc8c18d2700 1 -- 172.21.15.97:0/295723462 wait complete. 2023-10-08T09:54:24.199 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T09:54:24.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:54:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 -- 172.21.15.97:0/3249644564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10103a80 msgr2=0x7fca10105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 --2- 172.21.15.97:0/3249644564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10103a80 0x7fca10105e70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca04009990 tx=0x7fca04009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 -- 172.21.15.97:0/3249644564 shutdown_connections 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 --2- 172.21.15.97:0/3249644564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca101063b0 0x7fca10108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 --2- 172.21.15.97:0/3249644564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca10101150 0x7fca10103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 --2- 172.21.15.97:0/3249644564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10103a80 0x7fca10105e70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 -- 172.21.15.97:0/3249644564 >> 172.21.15.97:0/3249644564 conn(0x7fca100fab40 msgr2=0x7fca100fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 -- 172.21.15.97:0/3249644564 shutdown_connections 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.321+0000 7fca1583a700 1 -- 172.21.15.97:0/3249644564 wait complete. 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 Processor -- start 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 -- start start 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca10103a80 0x7fca10196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca101063b0 0x7fca1019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1019e0b0 con 0x7fca10101150 2023-10-08T09:54:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1019e1f0 con 0x7fca10103a80 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca1583a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1019e380 con 0x7fca101063b0 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.322+0000 7fca0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55582/0 (socket says 172.21.15.97:55582) 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 -- 172.21.15.97:0/2768852966 learned_addr learned my addr 172.21.15.97:0/2768852966 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0f7fe700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca101063b0 0x7fca1019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0e7fc700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca10103a80 0x7fca10196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 -- 172.21.15.97:0/2768852966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca101063b0 msgr2=0x7fca1019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca101063b0 0x7fca1019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 -- 172.21.15.97:0/2768852966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca10103a80 msgr2=0x7fca10196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca10103a80 0x7fca10196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 -- 172.21.15.97:0/2768852966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040095f0 con 0x7fca10101150 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca0effd700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c9b0 tx=0x7fca0000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00007b00 con 0x7fca10101150 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1019e660 con 0x7fca10101150 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca00007c60 con 0x7fca10101150 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00018660 con 0x7fca10101150 2023-10-08T09:54:24.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.323+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1019eb70 con 0x7fca10101150 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.324+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca100fc6c0 con 0x7fca10101150 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.325+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fca0000f450 con 0x7fca10101150 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.326+0000 7fca14838700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 0x7fc9f80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.326+0000 7fca0e7fc700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 0x7fc9f80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.326+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fca000ccf00 con 0x7fca10101150 2023-10-08T09:54:24.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.326+0000 7fca0e7fc700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 0x7fc9f80a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca04006010 tx=0x7fca040037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.330+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fca00096310 con 0x7fca10101150 2023-10-08T09:54:24.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:active 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T09:54:24.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:24 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:active 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T09:54:24.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:24 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3433325578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] up:active 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T09:54:24.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T09:54:24.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:24 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:54:24.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.566+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca1010c1b0 con 0x7fca10101150 2023-10-08T09:54:24.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.566+0000 7fca14838700 1 -- 172.21.15.97:0/2768852966 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7fca00099930 con 0x7fca10101150 2023-10-08T09:54:24.567 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:24.567 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:54:24.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 msgr2=0x7fc9f80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 0x7fc9f80a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca04006010 tx=0x7fca040037b0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 msgr2=0x7fca10196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c9b0 tx=0x7fca0000ccc0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 shutdown_connections 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc9f80a1480 0x7fc9f80a3940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca101063b0 0x7fca1019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10101150 0x7fca10196580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 --2- 172.21.15.97:0/2768852966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca10103a80 0x7fca10196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 >> 172.21.15.97:0/2768852966 conn(0x7fca100fab40 msgr2=0x7fca10101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 shutdown_connections 2023-10-08T09:54:24.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.569+0000 7fca1583a700 1 -- 172.21.15.97:0/2768852966 wait complete. 2023-10-08T09:54:24.576 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:54:24.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:54:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 -- 172.21.15.97:0/3171073141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e28100790 msgr2=0x7f4e28100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/3171073141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e28100790 0x7f4e28100bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4e180099c0 tx=0x7f4e18009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 -- 172.21.15.97:0/3171073141 shutdown_connections 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/3171073141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28101130 0x7f4e281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/3171073141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e280ff5a0 0x7f4e280ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/3171073141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e28100790 0x7f4e28100bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.701+0000 7f4e2f81a700 1 -- 172.21.15.97:0/3171073141 >> 172.21.15.97:0/3171073141 conn(0x7f4e280fab10 msgr2=0x7f4e280fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- 172.21.15.97:0/3171073141 shutdown_connections 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- 172.21.15.97:0/3171073141 wait complete. 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 Processor -- start 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- start start 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28100790 0x7f4e28196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e28101130 0x7f4e2819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e020 con 0x7f4e280ff5a0 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e160 con 0x7f4e28100790 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2f81a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e2f0 con 0x7f4e28101130 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2d5b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2d5b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55596/0 (socket says 172.21.15.97:55596) 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2d5b6700 1 -- 172.21.15.97:0/2882045827 learned_addr learned my addr 172.21.15.97:0/2882045827 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.702+0000 7f4e2cdb5700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28100790 0x7f4e28196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2ddb7700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e28101130 0x7f4e2819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 -- 172.21.15.97:0/2882045827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e28101130 msgr2=0x7f4e2819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e28101130 0x7f4e2819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 -- 172.21.15.97:0/2882045827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28100790 msgr2=0x7f4e28196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28100790 0x7f4e28196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 -- 172.21.15.97:0/2882045827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180095f0 con 0x7f4e280ff5a0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2d5b6700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c990 tx=0x7f4e2400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24007b00 con 0x7f4e280ff5a0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e24007c60 con 0x7f4e280ff5a0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2819e5d0 con 0x7f4e280ff5a0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24018660 con 0x7f4e280ff5a0 2023-10-08T09:54:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.703+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2819eae0 con 0x7f4e280ff5a0 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.704+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28067450 con 0x7f4e280ff5a0 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.705+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4e2400f450 con 0x7f4e280ff5a0 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.706+0000 7f4e1e7fc700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 0x7f4e140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.706+0000 7f4e2cdb5700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 0x7f4e140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.706+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e240ccf50 con 0x7f4e280ff5a0 2023-10-08T09:54:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.706+0000 7f4e2cdb5700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 0x7f4e140a3940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.709+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e240963f0 con 0x7f4e280ff5a0 2023-10-08T09:54:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.901+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e2810adc0 con 0x7f4e280ff5a0 2023-10-08T09:54:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.903+0000 7f4e1e7fc700 1 -- 172.21.15.97:0/2882045827 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7f4e24099a10 con 0x7f4e280ff5a0 2023-10-08T09:54:24.904 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:24.904 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 msgr2=0x7f4e140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 0x7f4e140a3940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18000bc0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 msgr2=0x7f4e281964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c990 tx=0x7f4e2400cca0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 shutdown_connections 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e140a1480 0x7f4e140a3940 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e28101130 0x7f4e2819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e280ff5a0 0x7f4e281964f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 --2- 172.21.15.97:0/2882045827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e28100790 0x7f4e28196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:24.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 >> 172.21.15.97:0/2882045827 conn(0x7f4e280fab10 msgr2=0x7f4e28067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:24.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 shutdown_connections 2023-10-08T09:54:24.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:24.905+0000 7f4e2f81a700 1 -- 172.21.15.97:0/2882045827 wait complete. 2023-10-08T09:54:24.919 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T09:54:24.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:54:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/3770167957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100b60 msgr2=0x7f9b48100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/3770167957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100b60 0x7f9b48100f80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009990 tx=0x7f9b3c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/3770167957 shutdown_connections 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/3770167957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100230 0x7f9b48107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/3770167957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100b60 0x7f9b48100f80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/3770167957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 0x7f9b480ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/3770167957 >> 172.21.15.97:0/3770167957 conn(0x7f9b480fa2c0 msgr2=0x7f9b480fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/3770167957 shutdown_connections 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.038+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/3770167957 wait complete. 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 Processor -- start 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 -- start start 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 0x7f9b48195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100b60 0x7f9b4819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4819d060 con 0x7f9b48100230 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4819d1a0 con 0x7f9b480ff890 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4cc4d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4819d330 con 0x7f9b48100b60 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b45d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b45d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55612/0 (socket says 172.21.15.97:55612) 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b45d9b700 1 -- 172.21.15.97:0/2794738092 learned_addr learned my addr 172.21.15.97:0/2794738092 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.039+0000 7f9b4659c700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 0x7f9b48195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b46d9d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100b60 0x7f9b4819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 -- 172.21.15.97:0/2794738092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100b60 msgr2=0x7f9b4819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100b60 0x7f9b4819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 -- 172.21.15.97:0/2794738092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 msgr2=0x7f9b48195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 0x7f9b48195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 -- 172.21.15.97:0/2794738092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0095f0 con 0x7f9b48100230 2023-10-08T09:54:25.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b45d9b700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800d950 tx=0x7f9b3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38009940 con 0x7f9b48100230 2023-10-08T09:54:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4819d610 con 0x7f9b48100230 2023-10-08T09:54:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b38010460 con 0x7f9b48100230 2023-10-08T09:54:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800f620 con 0x7f9b48100230 2023-10-08T09:54:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.040+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4819db70 con 0x7f9b48100230 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.041+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48106b50 con 0x7f9b48100230 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.042+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9b380108d0 con 0x7f9b48100230 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.042+0000 7f9b377fe700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 0x7f9b300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.043+0000 7f9b4659c700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 0x7f9b300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.043+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9b380ccdc0 con 0x7f9b48100230 2023-10-08T09:54:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.043+0000 7f9b4659c700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 0x7f9b300a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00b5c0 tx=0x7f9b3c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:25.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.047+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9b380961d0 con 0x7f9b48100230 2023-10-08T09:54:25.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.237+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b4802cf60 con 0x7f9b300a1480 2023-10-08T09:54:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.251+0000 7f9b377fe700 1 -- 172.21.15.97:0/2794738092 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423186 (secure 0 0 0) 0x7f9b4802cf60 con 0x7f9b300a1480 2023-10-08T09:54:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 msgr2=0x7f9b300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 0x7f9b300a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00b5c0 tx=0x7f9b3c003790 comp rx=0 tx=0).stop 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 msgr2=0x7f9b48195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800d950 tx=0x7f9b3800dc60 comp rx=0 tx=0).stop 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 shutdown_connections 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b300a1480 0x7f9b300a3940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b48100b60 0x7f9b4819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b480ff890 0x7f9b48195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 --2- 172.21.15.97:0/2794738092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b48100230 0x7f9b48195a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.255+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 >> 172.21.15.97:0/2794738092 conn(0x7f9b480fa2c0 msgr2=0x7f9b48103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.256+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 shutdown_connections 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.256+0000 7f9b4cc4d700 1 -- 172.21.15.97:0/2794738092 wait complete. 2023-10-08T09:54:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:54:25.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-10-08T09:54:23.881110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595098,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":237,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337028,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604796,"statfs":{"total":1151940427776,"available":1151595311104,"internally_reserved":0,"allocated":13094912,"data_stored":7261224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":231,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236714+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:01.236714+0000","last_peered":"2023-10-08T09:54:01.236714+0000","last_clean":"2023-10-08T09:54:01.236714+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:01.236714+0000","last_undegraded":"2023-10-08T09:54:01.236714+0000","last_fullsized":"2023-10-08T09:54:01.236714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233407+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:01.233407+0000","last_peered":"2023-10-08T09:54:01.233407+0000","last_clean":"2023-10-08T09:54:01.233407+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:01.233407+0000","last_undegraded":"2023-10-08T09:54:01.233407+0000","last_fullsized":"2023-10-08T09:54:01.233407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234233+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:01.234233+0000","last_peered":"2023-10-08T09:54:01.234233+0000","last_clean":"2023-10-08T09:54:01.234233+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:01.234233+0000","last_undegraded":"2023-10-08T09:54:01.234233+0000","last_fullsized":"2023-10-08T09:54:01.234233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236452+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:01.236452+0000","last_peered":"2023-10-08T09:54:01.236452+0000","last_clean":"2023-10-08T09:54:01.236452+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:01.236452+0000","last_undegraded":"2023-10-08T09:54:01.236452+0000","last_fullsized":"2023-10-08T09:54:01.236452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227791+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:01.227791+0000","last_peered":"2023-10-08T09:54:01.227791+0000","last_clean":"2023-10-08T09:54:01.227791+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:01.227791+0000","last_undegraded":"2023-10-08T09:54:01.227791+0000","last_fullsized":"2023-10-08T09:54:01.227791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303490+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:01.303490+0000","last_peered":"2023-10-08T09:54:01.303490+0000","last_clean":"2023-10-08T09:54:01.303490+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:01.303490+0000","last_undegraded":"2023-10-08T09:54:01.303490+0000","last_fullsized":"2023-10-08T09:54:01.303490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234585+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:01.234585+0000","last_peered":"2023-10-08T09:54:01.234585+0000","last_clean":"2023-10-08T09:54:01.234585+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:01.234585+0000","last_undegraded":"2023-10-08T09:54:01.234585+0000","last_fullsized":"2023-10-08T09:54:01.234585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235130+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:01.235130+0000","last_peered":"2023-10-08T09:54:01.235130+0000","last_clean":"2023-10-08T09:54:01.235130+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:01.235130+0000","last_undegraded":"2023-10-08T09:54:01.235130+0000","last_fullsized":"2023-10-08T09:54:01.235130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228339+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:54:01.228339+0000","last_peered":"2023-10-08T09:54:01.228339+0000","last_clean":"2023-10-08T09:54:01.228339+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:54:01.228339+0000","last_undegraded":"2023-10-08T09:54:01.228339+0000","last_fullsized":"2023-10-08T09:54:01.228339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303360+0000","last_change":"2023-10-08T09:53:56.901387+0000","last_active":"2023-10-08T09:54:01.303360+0000","last_peered":"2023-10-08T09:54:01.303360+0000","last_clean":"2023-10-08T09:54:01.303360+0000","last_became_active":"2023-10-08T09:53:56.901091+0000","last_became_peered":"2023-10-08T09:53:56.901091+0000","last_unstale":"2023-10-08T09:54:01.303360+0000","last_undegraded":"2023-10-08T09:54:01.303360+0000","last_fullsized":"2023-10-08T09:54:01.303360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:01.441142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235763+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:01.235763+0000","last_peered":"2023-10-08T09:54:01.235763+0000","last_clean":"2023-10-08T09:54:01.235763+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:01.235763+0000","last_undegraded":"2023-10-08T09:54:01.235763+0000","last_fullsized":"2023-10-08T09:54:01.235763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236350+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:01.236350+0000","last_peered":"2023-10-08T09:54:01.236350+0000","last_clean":"2023-10-08T09:54:01.236350+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:01.236350+0000","last_undegraded":"2023-10-08T09:54:01.236350+0000","last_fullsized":"2023-10-08T09:54:01.236350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252372+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:01.252372+0000","last_peered":"2023-10-08T09:54:01.252372+0000","last_clean":"2023-10-08T09:54:01.252372+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:01.252372+0000","last_undegraded":"2023-10-08T09:54:01.252372+0000","last_fullsized":"2023-10-08T09:54:01.252372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233417+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:01.233417+0000","last_peered":"2023-10-08T09:54:01.233417+0000","last_clean":"2023-10-08T09:54:01.233417+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:01.233417+0000","last_undegraded":"2023-10-08T09:54:01.233417+0000","last_fullsized":"2023-10-08T09:54:01.233417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233696+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:01.233696+0000","last_peered":"2023-10-08T09:54:01.233696+0000","last_clean":"2023-10-08T09:54:01.233696+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:01.233696+0000","last_undegraded":"2023-10-08T09:54:01.233696+0000","last_fullsized":"2023-10-08T09:54:01.233696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227867+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:54:01.227867+0000","last_peered":"2023-10-08T09:54:01.227867+0000","last_clean":"2023-10-08T09:54:01.227867+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:54:01.227867+0000","last_undegraded":"2023-10-08T09:54:01.227867+0000","last_fullsized":"2023-10-08T09:54:01.227867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236625+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:01.236625+0000","last_peered":"2023-10-08T09:54:01.236625+0000","last_clean":"2023-10-08T09:54:01.236625+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:01.236625+0000","last_undegraded":"2023-10-08T09:54:01.236625+0000","last_fullsized":"2023-10-08T09:54:01.236625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241845+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:01.241845+0000","last_peered":"2023-10-08T09:54:01.241845+0000","last_clean":"2023-10-08T09:54:01.241845+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:01.241845+0000","last_undegraded":"2023-10-08T09:54:01.241845+0000","last_fullsized":"2023-10-08T09:54:01.241845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234927+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:01.234927+0000","last_peered":"2023-10-08T09:54:01.234927+0000","last_clean":"2023-10-08T09:54:01.234927+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:01.234927+0000","last_undegraded":"2023-10-08T09:54:01.234927+0000","last_fullsized":"2023-10-08T09:54:01.234927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242614+0000","last_change":"2023-10-08T09:54:01.240656+0000","last_active":"2023-10-08T09:54:02.242614+0000","last_peered":"2023-10-08T09:54:02.242614+0000","last_clean":"2023-10-08T09:54:02.242614+0000","last_became_active":"2023-10-08T09:54:01.240274+0000","last_became_peered":"2023-10-08T09:54:01.240274+0000","last_unstale":"2023-10-08T09:54:02.242614+0000","last_undegraded":"2023-10-08T09:54:02.242614+0000","last_fullsized":"2023-10-08T09:54:02.242614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:14.800643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241838+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:01.241838+0000","last_peered":"2023-10-08T09:54:01.241838+0000","last_clean":"2023-10-08T09:54:01.241838+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:01.241838+0000","last_undegraded":"2023-10-08T09:54:01.241838+0000","last_fullsized":"2023-10-08T09:54:01.241838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241219+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:01.241219+0000","last_peered":"2023-10-08T09:54:01.241219+0000","last_clean":"2023-10-08T09:54:01.241219+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:01.241219+0000","last_undegraded":"2023-10-08T09:54:01.241219+0000","last_fullsized":"2023-10-08T09:54:01.241219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502709+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:03.502709+0000","last_peered":"2023-10-08T09:54:03.502709+0000","last_clean":"2023-10-08T09:54:03.502709+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:03.502709+0000","last_undegraded":"2023-10-08T09:54:03.502709+0000","last_fullsized":"2023-10-08T09:54:03.502709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401922+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:02.401922+0000","last_peered":"2023-10-08T09:54:02.401922+0000","last_clean":"2023-10-08T09:54:02.401922+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:02.401922+0000","last_undegraded":"2023-10-08T09:54:02.401922+0000","last_fullsized":"2023-10-08T09:54:02.401922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233799+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:01.233799+0000","last_peered":"2023-10-08T09:54:01.233799+0000","last_clean":"2023-10-08T09:54:01.233799+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:01.233799+0000","last_undegraded":"2023-10-08T09:54:01.233799+0000","last_fullsized":"2023-10-08T09:54:01.233799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502990+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:03.502990+0000","last_peered":"2023-10-08T09:54:03.502990+0000","last_clean":"2023-10-08T09:54:03.502990+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:03.502990+0000","last_undegraded":"2023-10-08T09:54:03.502990+0000","last_fullsized":"2023-10-08T09:54:03.502990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228305+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:01.228305+0000","last_peered":"2023-10-08T09:54:01.228305+0000","last_clean":"2023-10-08T09:54:01.228305+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:01.228305+0000","last_undegraded":"2023-10-08T09:54:01.228305+0000","last_fullsized":"2023-10-08T09:54:01.228305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228030+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:54:01.228030+0000","last_peered":"2023-10-08T09:54:01.228030+0000","last_clean":"2023-10-08T09:54:01.228030+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:54:01.228030+0000","last_undegraded":"2023-10-08T09:54:01.228030+0000","last_fullsized":"2023-10-08T09:54:01.228030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502747+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:03.502747+0000","last_peered":"2023-10-08T09:54:03.502747+0000","last_clean":"2023-10-08T09:54:03.502747+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:03.502747+0000","last_undegraded":"2023-10-08T09:54:03.502747+0000","last_fullsized":"2023-10-08T09:54:03.502747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228285+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:01.228285+0000","last_peered":"2023-10-08T09:54:01.228285+0000","last_clean":"2023-10-08T09:54:01.228285+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:01.228285+0000","last_undegraded":"2023-10-08T09:54:01.228285+0000","last_fullsized":"2023-10-08T09:54:01.228285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230787+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:01.230787+0000","last_peered":"2023-10-08T09:54:01.230787+0000","last_clean":"2023-10-08T09:54:01.230787+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:01.230787+0000","last_undegraded":"2023-10-08T09:54:01.230787+0000","last_fullsized":"2023-10-08T09:54:01.230787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243324+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:02.243324+0000","last_peered":"2023-10-08T09:54:02.243324+0000","last_clean":"2023-10-08T09:54:02.243324+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:02.243324+0000","last_undegraded":"2023-10-08T09:54:02.243324+0000","last_fullsized":"2023-10-08T09:54:02.243324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230400+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:54:01.230400+0000","last_peered":"2023-10-08T09:54:01.230400+0000","last_clean":"2023-10-08T09:54:01.230400+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:54:01.230400+0000","last_undegraded":"2023-10-08T09:54:01.230400+0000","last_fullsized":"2023-10-08T09:54:01.230400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228580+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:01.228580+0000","last_peered":"2023-10-08T09:54:01.228580+0000","last_clean":"2023-10-08T09:54:01.228580+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:01.228580+0000","last_undegraded":"2023-10-08T09:54:01.228580+0000","last_fullsized":"2023-10-08T09:54:01.228580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243923+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:02.243923+0000","last_peered":"2023-10-08T09:54:02.243923+0000","last_clean":"2023-10-08T09:54:02.243923+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:02.243923+0000","last_undegraded":"2023-10-08T09:54:02.243923+0000","last_fullsized":"2023-10-08T09:54:02.243923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.302998+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:01.302998+0000","last_peered":"2023-10-08T09:54:01.302998+0000","last_clean":"2023-10-08T09:54:01.302998+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:01.302998+0000","last_undegraded":"2023-10-08T09:54:01.302998+0000","last_fullsized":"2023-10-08T09:54:01.302998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230447+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:01.230447+0000","last_peered":"2023-10-08T09:54:01.230447+0000","last_clean":"2023-10-08T09:54:01.230447+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:01.230447+0000","last_undegraded":"2023-10-08T09:54:01.230447+0000","last_fullsized":"2023-10-08T09:54:01.230447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502863+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:03.502863+0000","last_peered":"2023-10-08T09:54:03.502863+0000","last_clean":"2023-10-08T09:54:03.502863+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:03.502863+0000","last_undegraded":"2023-10-08T09:54:03.502863+0000","last_fullsized":"2023-10-08T09:54:03.502863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252294+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:01.252294+0000","last_peered":"2023-10-08T09:54:01.252294+0000","last_clean":"2023-10-08T09:54:01.252294+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:01.252294+0000","last_undegraded":"2023-10-08T09:54:01.252294+0000","last_fullsized":"2023-10-08T09:54:01.252294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235096+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:01.235096+0000","last_peered":"2023-10-08T09:54:01.235096+0000","last_clean":"2023-10-08T09:54:01.235096+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:01.235096+0000","last_undegraded":"2023-10-08T09:54:01.235096+0000","last_fullsized":"2023-10-08T09:54:01.235096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502710+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:03.502710+0000","last_peered":"2023-10-08T09:54:03.502710+0000","last_clean":"2023-10-08T09:54:03.502710+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:03.502710+0000","last_undegraded":"2023-10-08T09:54:03.502710+0000","last_fullsized":"2023-10-08T09:54:03.502710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230271+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:01.230271+0000","last_peered":"2023-10-08T09:54:01.230271+0000","last_clean":"2023-10-08T09:54:01.230271+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:01.230271+0000","last_undegraded":"2023-10-08T09:54:01.230271+0000","last_fullsized":"2023-10-08T09:54:01.230271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240133+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:01.240133+0000","last_peered":"2023-10-08T09:54:01.240133+0000","last_clean":"2023-10-08T09:54:01.240133+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:01.240133+0000","last_undegraded":"2023-10-08T09:54:01.240133+0000","last_fullsized":"2023-10-08T09:54:01.240133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401534+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:02.401534+0000","last_peered":"2023-10-08T09:54:02.401534+0000","last_clean":"2023-10-08T09:54:02.401534+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:02.401534+0000","last_undegraded":"2023-10-08T09:54:02.401534+0000","last_fullsized":"2023-10-08T09:54:02.401534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303177+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:54:01.303177+0000","last_peered":"2023-10-08T09:54:01.303177+0000","last_clean":"2023-10-08T09:54:01.303177+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:54:01.303177+0000","last_undegraded":"2023-10-08T09:54:01.303177+0000","last_fullsized":"2023-10-08T09:54:01.303177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228327+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:54:01.228327+0000","last_peered":"2023-10-08T09:54:01.228327+0000","last_clean":"2023-10-08T09:54:01.228327+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:54:01.228327+0000","last_undegraded":"2023-10-08T09:54:01.228327+0000","last_fullsized":"2023-10-08T09:54:01.228327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.501874+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:03.501874+0000","last_peered":"2023-10-08T09:54:03.501874+0000","last_clean":"2023-10-08T09:54:03.501874+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:03.501874+0000","last_undegraded":"2023-10-08T09:54:03.501874+0000","last_fullsized":"2023-10-08T09:54:03.501874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235442+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:01.235442+0000","last_peered":"2023-10-08T09:54:01.235442+0000","last_clean":"2023-10-08T09:54:01.235442+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:01.235442+0000","last_undegraded":"2023-10-08T09:54:01.235442+0000","last_fullsized":"2023-10-08T09:54:01.235442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233693+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:01.233693+0000","last_peered":"2023-10-08T09:54:01.233693+0000","last_clean":"2023-10-08T09:54:01.233693+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:01.233693+0000","last_undegraded":"2023-10-08T09:54:01.233693+0000","last_fullsized":"2023-10-08T09:54:01.233693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503155+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:03.503155+0000","last_peered":"2023-10-08T09:54:03.503155+0000","last_clean":"2023-10-08T09:54:03.503155+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:03.503155+0000","last_undegraded":"2023-10-08T09:54:03.503155+0000","last_fullsized":"2023-10-08T09:54:03.503155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230618+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:01.230618+0000","last_peered":"2023-10-08T09:54:01.230618+0000","last_clean":"2023-10-08T09:54:01.230618+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:01.230618+0000","last_undegraded":"2023-10-08T09:54:01.230618+0000","last_fullsized":"2023-10-08T09:54:01.230618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228133+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:54:01.228133+0000","last_peered":"2023-10-08T09:54:01.228133+0000","last_clean":"2023-10-08T09:54:01.228133+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:54:01.228133+0000","last_undegraded":"2023-10-08T09:54:01.228133+0000","last_fullsized":"2023-10-08T09:54:01.228133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502416+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:03.502416+0000","last_peered":"2023-10-08T09:54:03.502416+0000","last_clean":"2023-10-08T09:54:03.502416+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:03.502416+0000","last_undegraded":"2023-10-08T09:54:03.502416+0000","last_fullsized":"2023-10-08T09:54:03.502416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'10","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:13.023680+0000","last_change":"2023-10-08T09:53:56.203103+0000","last_active":"2023-10-08T09:54:13.023680+0000","last_peered":"2023-10-08T09:54:13.023680+0000","last_clean":"2023-10-08T09:54:13.023680+0000","last_became_active":"2023-10-08T09:53:56.202807+0000","last_became_peered":"2023-10-08T09:53:56.202807+0000","last_unstale":"2023-10-08T09:54:13.023680+0000","last_undegraded":"2023-10-08T09:54:13.023680+0000","last_fullsized":"2023-10-08T09:54:13.023680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:01.154899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228142+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:01.228142+0000","last_peered":"2023-10-08T09:54:01.228142+0000","last_clean":"2023-10-08T09:54:01.228142+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:01.228142+0000","last_undegraded":"2023-10-08T09:54:01.228142+0000","last_fullsized":"2023-10-08T09:54:01.228142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243461+0000","last_change":"2023-10-08T09:54:01.251015+0000","last_active":"2023-10-08T09:54:02.243461+0000","last_peered":"2023-10-08T09:54:02.243461+0000","last_clean":"2023-10-08T09:54:02.243461+0000","last_became_active":"2023-10-08T09:54:01.250599+0000","last_became_peered":"2023-10-08T09:54:01.250599+0000","last_unstale":"2023-10-08T09:54:02.243461+0000","last_undegraded":"2023-10-08T09:54:02.243461+0000","last_fullsized":"2023-10-08T09:54:02.243461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:59.645562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234184+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:01.234184+0000","last_peered":"2023-10-08T09:54:01.234184+0000","last_clean":"2023-10-08T09:54:01.234184+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:01.234184+0000","last_undegraded":"2023-10-08T09:54:01.234184+0000","last_fullsized":"2023-10-08T09:54:01.234184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252351+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:01.252351+0000","last_peered":"2023-10-08T09:54:01.252351+0000","last_clean":"2023-10-08T09:54:01.252351+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:01.252351+0000","last_undegraded":"2023-10-08T09:54:01.252351+0000","last_fullsized":"2023-10-08T09:54:01.252351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502668+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:03.502668+0000","last_peered":"2023-10-08T09:54:03.502668+0000","last_clean":"2023-10-08T09:54:03.502668+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:03.502668+0000","last_undegraded":"2023-10-08T09:54:03.502668+0000","last_fullsized":"2023-10-08T09:54:03.502668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230348+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:01.230348+0000","last_peered":"2023-10-08T09:54:01.230348+0000","last_clean":"2023-10-08T09:54:01.230348+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:01.230348+0000","last_undegraded":"2023-10-08T09:54:01.230348+0000","last_fullsized":"2023-10-08T09:54:01.230348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236418+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:01.236418+0000","last_peered":"2023-10-08T09:54:01.236418+0000","last_clean":"2023-10-08T09:54:01.236418+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:01.236418+0000","last_undegraded":"2023-10-08T09:54:01.236418+0000","last_fullsized":"2023-10-08T09:54:01.236418+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502141+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:03.502141+0000","last_peered":"2023-10-08T09:54:03.502141+0000","last_clean":"2023-10-08T09:54:03.502141+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:03.502141+0000","last_undegraded":"2023-10-08T09:54:03.502141+0000","last_fullsized":"2023-10-08T09:54:03.502141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.232857+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:54:01.232857+0000","last_peered":"2023-10-08T09:54:01.232857+0000","last_clean":"2023-10-08T09:54:01.232857+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:54:01.232857+0000","last_undegraded":"2023-10-08T09:54:01.232857+0000","last_fullsized":"2023-10-08T09:54:01.232857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234983+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:01.234983+0000","last_peered":"2023-10-08T09:54:01.234983+0000","last_clean":"2023-10-08T09:54:01.234983+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:01.234983+0000","last_undegraded":"2023-10-08T09:54:01.234983+0000","last_fullsized":"2023-10-08T09:54:01.234983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.278175+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:02.278175+0000","last_peered":"2023-10-08T09:54:02.278175+0000","last_clean":"2023-10-08T09:54:02.278175+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:02.278175+0000","last_undegraded":"2023-10-08T09:54:02.278175+0000","last_fullsized":"2023-10-08T09:54:02.278175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227722+0000","last_change":"2023-10-08T09:53:56.200805+0000","last_active":"2023-10-08T09:54:01.227722+0000","last_peered":"2023-10-08T09:54:01.227722+0000","last_clean":"2023-10-08T09:54:01.227722+0000","last_became_active":"2023-10-08T09:53:56.200507+0000","last_became_peered":"2023-10-08T09:53:56.200507+0000","last_unstale":"2023-10-08T09:54:01.227722+0000","last_undegraded":"2023-10-08T09:54:01.227722+0000","last_fullsized":"2023-10-08T09:54:01.227722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.328356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230299+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:01.230299+0000","last_peered":"2023-10-08T09:54:01.230299+0000","last_clean":"2023-10-08T09:54:01.230299+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:01.230299+0000","last_undegraded":"2023-10-08T09:54:01.230299+0000","last_fullsized":"2023-10-08T09:54:01.230299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242768+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:02.242768+0000","last_peered":"2023-10-08T09:54:02.242768+0000","last_clean":"2023-10-08T09:54:02.242768+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:02.242768+0000","last_undegraded":"2023-10-08T09:54:02.242768+0000","last_fullsized":"2023-10-08T09:54:02.242768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228124+0000","last_change":"2023-10-08T09:53:56.199894+0000","last_active":"2023-10-08T09:54:01.228124+0000","last_peered":"2023-10-08T09:54:01.228124+0000","last_clean":"2023-10-08T09:54:01.228124+0000","last_became_active":"2023-10-08T09:53:56.199576+0000","last_became_peered":"2023-10-08T09:53:56.199576+0000","last_unstale":"2023-10-08T09:54:01.228124+0000","last_undegraded":"2023-10-08T09:54:01.228124+0000","last_fullsized":"2023-10-08T09:54:01.228124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:16.389646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235084+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:01.235084+0000","last_peered":"2023-10-08T09:54:01.235084+0000","last_clean":"2023-10-08T09:54:01.235084+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:01.235084+0000","last_undegraded":"2023-10-08T09:54:01.235084+0000","last_fullsized":"2023-10-08T09:54:01.235084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.590187+0000","last_change":"2023-10-08T09:54:01.304336+0000","last_active":"2023-10-08T09:54:03.590187+0000","last_peered":"2023-10-08T09:54:03.590187+0000","last_clean":"2023-10-08T09:54:03.590187+0000","last_became_active":"2023-10-08T09:54:01.303870+0000","last_became_peered":"2023-10-08T09:54:01.303870+0000","last_unstale":"2023-10-08T09:54:03.590187+0000","last_undegraded":"2023-10-08T09:54:03.590187+0000","last_fullsized":"2023-10-08T09:54:03.590187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:24.098993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240484+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:01.240484+0000","last_peered":"2023-10-08T09:54:01.240484+0000","last_clean":"2023-10-08T09:54:01.240484+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:01.240484+0000","last_undegraded":"2023-10-08T09:54:01.240484+0000","last_fullsized":"2023-10-08T09:54:01.240484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303050+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:01.303050+0000","last_peered":"2023-10-08T09:54:01.303050+0000","last_clean":"2023-10-08T09:54:01.303050+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:01.303050+0000","last_undegraded":"2023-10-08T09:54:01.303050+0000","last_fullsized":"2023-10-08T09:54:01.303050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.278006+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:02.278006+0000","last_peered":"2023-10-08T09:54:02.278006+0000","last_clean":"2023-10-08T09:54:02.278006+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:02.278006+0000","last_undegraded":"2023-10-08T09:54:02.278006+0000","last_fullsized":"2023-10-08T09:54:02.278006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234808+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:01.234808+0000","last_peered":"2023-10-08T09:54:01.234808+0000","last_clean":"2023-10-08T09:54:01.234808+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:01.234808+0000","last_undegraded":"2023-10-08T09:54:01.234808+0000","last_fullsized":"2023-10-08T09:54:01.234808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.302843+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:01.302843+0000","last_peered":"2023-10-08T09:54:01.302843+0000","last_clean":"2023-10-08T09:54:01.302843+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:01.302843+0000","last_undegraded":"2023-10-08T09:54:01.302843+0000","last_fullsized":"2023-10-08T09:54:01.302843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.277886+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:02.277886+0000","last_peered":"2023-10-08T09:54:02.277886+0000","last_clean":"2023-10-08T09:54:02.277886+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:02.277886+0000","last_undegraded":"2023-10-08T09:54:02.277886+0000","last_fullsized":"2023-10-08T09:54:02.277886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230726+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:01.230726+0000","last_peered":"2023-10-08T09:54:01.230726+0000","last_clean":"2023-10-08T09:54:01.230726+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:01.230726+0000","last_undegraded":"2023-10-08T09:54:01.230726+0000","last_fullsized":"2023-10-08T09:54:01.230726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252120+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:54:01.252120+0000","last_peered":"2023-10-08T09:54:01.252120+0000","last_clean":"2023-10-08T09:54:01.252120+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:54:01.252120+0000","last_undegraded":"2023-10-08T09:54:01.252120+0000","last_fullsized":"2023-10-08T09:54:01.252120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503294+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:03.503294+0000","last_peered":"2023-10-08T09:54:03.503294+0000","last_clean":"2023-10-08T09:54:03.503294+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:03.503294+0000","last_undegraded":"2023-10-08T09:54:03.503294+0000","last_fullsized":"2023-10-08T09:54:03.503294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240585+0000","last_change":"2023-10-08T09:53:56.887772+0000","last_active":"2023-10-08T09:54:01.240585+0000","last_peered":"2023-10-08T09:54:01.240585+0000","last_clean":"2023-10-08T09:54:01.240585+0000","last_became_active":"2023-10-08T09:53:56.887155+0000","last_became_peered":"2023-10-08T09:53:56.887155+0000","last_unstale":"2023-10-08T09:54:01.240585+0000","last_undegraded":"2023-10-08T09:54:01.240585+0000","last_fullsized":"2023-10-08T09:54:01.240585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:45.184494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234865+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:01.234865+0000","last_peered":"2023-10-08T09:54:01.234865+0000","last_clean":"2023-10-08T09:54:01.234865+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:01.234865+0000","last_undegraded":"2023-10-08T09:54:01.234865+0000","last_fullsized":"2023-10-08T09:54:01.234865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234778+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:01.234778+0000","last_peered":"2023-10-08T09:54:01.234778+0000","last_clean":"2023-10-08T09:54:01.234778+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:01.234778+0000","last_undegraded":"2023-10-08T09:54:01.234778+0000","last_fullsized":"2023-10-08T09:54:01.234778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243192+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:02.243192+0000","last_peered":"2023-10-08T09:54:02.243192+0000","last_clean":"2023-10-08T09:54:02.243192+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:02.243192+0000","last_undegraded":"2023-10-08T09:54:02.243192+0000","last_fullsized":"2023-10-08T09:54:02.243192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252114+0000","last_change":"2023-10-08T09:53:56.897985+0000","last_active":"2023-10-08T09:54:01.252114+0000","last_peered":"2023-10-08T09:54:01.252114+0000","last_clean":"2023-10-08T09:54:01.252114+0000","last_became_active":"2023-10-08T09:53:56.897520+0000","last_became_peered":"2023-10-08T09:53:56.897520+0000","last_unstale":"2023-10-08T09:54:01.252114+0000","last_undegraded":"2023-10-08T09:54:01.252114+0000","last_fullsized":"2023-10-08T09:54:01.252114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:52.733642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235448+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:01.235448+0000","last_peered":"2023-10-08T09:54:01.235448+0000","last_clean":"2023-10-08T09:54:01.235448+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:01.235448+0000","last_undegraded":"2023-10-08T09:54:01.235448+0000","last_fullsized":"2023-10-08T09:54:01.235448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502738+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:03.502738+0000","last_peered":"2023-10-08T09:54:03.502738+0000","last_clean":"2023-10-08T09:54:03.502738+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:03.502738+0000","last_undegraded":"2023-10-08T09:54:03.502738+0000","last_fullsized":"2023-10-08T09:54:03.502738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228431+0000","last_change":"2023-10-08T09:53:56.195815+0000","last_active":"2023-10-08T09:54:01.228431+0000","last_peered":"2023-10-08T09:54:01.228431+0000","last_clean":"2023-10-08T09:54:01.228431+0000","last_became_active":"2023-10-08T09:53:56.195472+0000","last_became_peered":"2023-10-08T09:53:56.195472+0000","last_unstale":"2023-10-08T09:54:01.228431+0000","last_undegraded":"2023-10-08T09:54:01.228431+0000","last_fullsized":"2023-10-08T09:54:01.228431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:53.341794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252674+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:01.252674+0000","last_peered":"2023-10-08T09:54:01.252674+0000","last_clean":"2023-10-08T09:54:01.252674+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:01.252674+0000","last_undegraded":"2023-10-08T09:54:01.252674+0000","last_fullsized":"2023-10-08T09:54:01.252674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401282+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:02.401282+0000","last_peered":"2023-10-08T09:54:02.401282+0000","last_clean":"2023-10-08T09:54:02.401282+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:02.401282+0000","last_undegraded":"2023-10-08T09:54:02.401282+0000","last_fullsized":"2023-10-08T09:54:02.401282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241933+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:01.241933+0000","last_peered":"2023-10-08T09:54:01.241933+0000","last_clean":"2023-10-08T09:54:01.241933+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:01.241933+0000","last_undegraded":"2023-10-08T09:54:01.241933+0000","last_fullsized":"2023-10-08T09:54:01.241933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240985+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:01.240985+0000","last_peered":"2023-10-08T09:54:01.240985+0000","last_clean":"2023-10-08T09:54:01.240985+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:01.240985+0000","last_undegraded":"2023-10-08T09:54:01.240985+0000","last_fullsized":"2023-10-08T09:54:01.240985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242332+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:02.242332+0000","last_peered":"2023-10-08T09:54:02.242332+0000","last_clean":"2023-10-08T09:54:02.242332+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:02.242332+0000","last_undegraded":"2023-10-08T09:54:02.242332+0000","last_fullsized":"2023-10-08T09:54:02.242332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233031+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:01.233031+0000","last_peered":"2023-10-08T09:54:01.233031+0000","last_clean":"2023-10-08T09:54:01.233031+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:01.233031+0000","last_undegraded":"2023-10-08T09:54:01.233031+0000","last_fullsized":"2023-10-08T09:54:01.233031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233493+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:01.233493+0000","last_peered":"2023-10-08T09:54:01.233493+0000","last_clean":"2023-10-08T09:54:01.233493+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:01.233493+0000","last_undegraded":"2023-10-08T09:54:01.233493+0000","last_fullsized":"2023-10-08T09:54:01.233493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503322+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:03.503322+0000","last_peered":"2023-10-08T09:54:03.503322+0000","last_clean":"2023-10-08T09:54:03.503322+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:03.503322+0000","last_undegraded":"2023-10-08T09:54:03.503322+0000","last_fullsized":"2023-10-08T09:54:03.503322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:22.181296+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:54:22.181296+0000","last_peered":"2023-10-08T09:54:22.181296+0000","last_clean":"2023-10-08T09:54:22.181296+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:54:22.181296+0000","last_undegraded":"2023-10-08T09:54:22.181296+0000","last_fullsized":"2023-10-08T09:54:22.181296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227731+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:01.227731+0000","last_peered":"2023-10-08T09:54:01.227731+0000","last_clean":"2023-10-08T09:54:01.227731+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:01.227731+0000","last_undegraded":"2023-10-08T09:54:01.227731+0000","last_fullsized":"2023-10-08T09:54:01.227731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502366+0000","last_change":"2023-10-08T09:54:01.246246+0000","last_active":"2023-10-08T09:54:03.502366+0000","last_peered":"2023-10-08T09:54:03.502366+0000","last_clean":"2023-10-08T09:54:03.502366+0000","last_became_active":"2023-10-08T09:54:01.245664+0000","last_became_peered":"2023-10-08T09:54:01.245664+0000","last_unstale":"2023-10-08T09:54:03.502366+0000","last_undegraded":"2023-10-08T09:54:03.502366+0000","last_fullsized":"2023-10-08T09:54:03.502366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:07.600486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:22.179798+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:22.179798+0000","last_peered":"2023-10-08T09:54:22.179798+0000","last_clean":"2023-10-08T09:54:22.179798+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:22.179798+0000","last_undegraded":"2023-10-08T09:54:22.179798+0000","last_fullsized":"2023-10-08T09:54:22.179798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234272+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:01.234272+0000","last_peered":"2023-10-08T09:54:01.234272+0000","last_clean":"2023-10-08T09:54:01.234272+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:01.234272+0000","last_undegraded":"2023-10-08T09:54:01.234272+0000","last_fullsized":"2023-10-08T09:54:01.234272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503449+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:03.503449+0000","last_peered":"2023-10-08T09:54:03.503449+0000","last_clean":"2023-10-08T09:54:03.503449+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:03.503449+0000","last_undegraded":"2023-10-08T09:54:03.503449+0000","last_fullsized":"2023-10-08T09:54:03.503449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233991+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:01.233991+0000","last_peered":"2023-10-08T09:54:01.233991+0000","last_clean":"2023-10-08T09:54:01.233991+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:01.233991+0000","last_undegraded":"2023-10-08T09:54:01.233991+0000","last_fullsized":"2023-10-08T09:54:01.233991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240781+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:01.240781+0000","last_peered":"2023-10-08T09:54:01.240781+0000","last_clean":"2023-10-08T09:54:01.240781+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:01.240781+0000","last_undegraded":"2023-10-08T09:54:01.240781+0000","last_fullsized":"2023-10-08T09:54:01.240781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503718+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:03.503718+0000","last_peered":"2023-10-08T09:54:03.503718+0000","last_clean":"2023-10-08T09:54:03.503718+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:03.503718+0000","last_undegraded":"2023-10-08T09:54:03.503718+0000","last_fullsized":"2023-10-08T09:54:03.503718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227921+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:01.227921+0000","last_peered":"2023-10-08T09:54:01.227921+0000","last_clean":"2023-10-08T09:54:01.227921+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:01.227921+0000","last_undegraded":"2023-10-08T09:54:01.227921+0000","last_fullsized":"2023-10-08T09:54:01.227921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234986+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:01.234986+0000","last_peered":"2023-10-08T09:54:01.234986+0000","last_clean":"2023-10-08T09:54:01.234986+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:01.234986+0000","last_undegraded":"2023-10-08T09:54:01.234986+0000","last_fullsized":"2023-10-08T09:54:01.234986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.624263+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:54:02.624263+0000","last_peered":"2023-10-08T09:54:02.624263+0000","last_clean":"2023-10-08T09:54:02.624263+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:54:02.624263+0000","last_undegraded":"2023-10-08T09:54:02.624263+0000","last_fullsized":"2023-10-08T09:54:02.624263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234654+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:01.234654+0000","last_peered":"2023-10-08T09:54:01.234654+0000","last_clean":"2023-10-08T09:54:01.234654+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:01.234654+0000","last_undegraded":"2023-10-08T09:54:01.234654+0000","last_fullsized":"2023-10-08T09:54:01.234654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235314+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:01.235314+0000","last_peered":"2023-10-08T09:54:01.235314+0000","last_clean":"2023-10-08T09:54:01.235314+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:01.235314+0000","last_undegraded":"2023-10-08T09:54:01.235314+0000","last_fullsized":"2023-10-08T09:54:01.235314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.501786+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:03.501786+0000","last_peered":"2023-10-08T09:54:03.501786+0000","last_clean":"2023-10-08T09:54:03.501786+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:03.501786+0000","last_undegraded":"2023-10-08T09:54:03.501786+0000","last_fullsized":"2023-10-08T09:54:03.501786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230841+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:01.230841+0000","last_peered":"2023-10-08T09:54:01.230841+0000","last_clean":"2023-10-08T09:54:01.230841+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:01.230841+0000","last_undegraded":"2023-10-08T09:54:01.230841+0000","last_fullsized":"2023-10-08T09:54:01.230841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235109+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:01.235109+0000","last_peered":"2023-10-08T09:54:01.235109+0000","last_clean":"2023-10-08T09:54:01.235109+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:01.235109+0000","last_undegraded":"2023-10-08T09:54:01.235109+0000","last_fullsized":"2023-10-08T09:54:01.235109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243323+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:02.243323+0000","last_peered":"2023-10-08T09:54:02.243323+0000","last_clean":"2023-10-08T09:54:02.243323+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:02.243323+0000","last_undegraded":"2023-10-08T09:54:02.243323+0000","last_fullsized":"2023-10-08T09:54:02.243323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252739+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:01.252739+0000","last_peered":"2023-10-08T09:54:01.252739+0000","last_clean":"2023-10-08T09:54:01.252739+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:01.252739+0000","last_undegraded":"2023-10-08T09:54:01.252739+0000","last_fullsized":"2023-10-08T09:54:01.252739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235451+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:01.235451+0000","last_peered":"2023-10-08T09:54:01.235451+0000","last_clean":"2023-10-08T09:54:01.235451+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:01.235451+0000","last_undegraded":"2023-10-08T09:54:01.235451+0000","last_fullsized":"2023-10-08T09:54:01.235451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234365+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:01.234365+0000","last_peered":"2023-10-08T09:54:01.234365+0000","last_clean":"2023-10-08T09:54:01.234365+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:01.234365+0000","last_undegraded":"2023-10-08T09:54:01.234365+0000","last_fullsized":"2023-10-08T09:54:01.234365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234047+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:01.234047+0000","last_peered":"2023-10-08T09:54:01.234047+0000","last_clean":"2023-10-08T09:54:01.234047+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:01.234047+0000","last_undegraded":"2023-10-08T09:54:01.234047+0000","last_fullsized":"2023-10-08T09:54:01.234047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233886+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:01.233886+0000","last_peered":"2023-10-08T09:54:01.233886+0000","last_clean":"2023-10-08T09:54:01.233886+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:01.233886+0000","last_undegraded":"2023-10-08T09:54:01.233886+0000","last_fullsized":"2023-10-08T09:54:01.233886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234060+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:01.234060+0000","last_peered":"2023-10-08T09:54:01.234060+0000","last_clean":"2023-10-08T09:54:01.234060+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:01.234060+0000","last_undegraded":"2023-10-08T09:54:01.234060+0000","last_fullsized":"2023-10-08T09:54:01.234060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252584+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:01.252584+0000","last_peered":"2023-10-08T09:54:01.252584+0000","last_clean":"2023-10-08T09:54:01.252584+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:01.252584+0000","last_undegraded":"2023-10-08T09:54:01.252584+0000","last_fullsized":"2023-10-08T09:54:01.252584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235741+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:01.235741+0000","last_peered":"2023-10-08T09:54:01.235741+0000","last_clean":"2023-10-08T09:54:01.235741+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:01.235741+0000","last_undegraded":"2023-10-08T09:54:01.235741+0000","last_fullsized":"2023-10-08T09:54:01.235741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235051+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:01.235051+0000","last_peered":"2023-10-08T09:54:01.235051+0000","last_clean":"2023-10-08T09:54:01.235051+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:01.235051+0000","last_undegraded":"2023-10-08T09:54:01.235051+0000","last_fullsized":"2023-10-08T09:54:01.235051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241060+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:01.241060+0000","last_peered":"2023-10-08T09:54:01.241060+0000","last_clean":"2023-10-08T09:54:01.241060+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:01.241060+0000","last_undegraded":"2023-10-08T09:54:01.241060+0000","last_fullsized":"2023-10-08T09:54:01.241060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230165+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:01.230165+0000","last_peered":"2023-10-08T09:54:01.230165+0000","last_clean":"2023-10-08T09:54:01.230165+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:01.230165+0000","last_undegraded":"2023-10-08T09:54:01.230165+0000","last_fullsized":"2023-10-08T09:54:01.230165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.239394+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:01.239394+0000","last_peered":"2023-10-08T09:54:01.239394+0000","last_clean":"2023-10-08T09:54:01.239394+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:01.239394+0000","last_undegraded":"2023-10-08T09:54:01.239394+0000","last_fullsized":"2023-10-08T09:54:01.239394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228627+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:54:01.228627+0000","last_peered":"2023-10-08T09:54:01.228627+0000","last_clean":"2023-10-08T09:54:01.228627+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:54:01.228627+0000","last_undegraded":"2023-10-08T09:54:01.228627+0000","last_fullsized":"2023-10-08T09:54:01.228627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252458+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:01.252458+0000","last_peered":"2023-10-08T09:54:01.252458+0000","last_clean":"2023-10-08T09:54:01.252458+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:01.252458+0000","last_undegraded":"2023-10-08T09:54:01.252458+0000","last_fullsized":"2023-10-08T09:54:01.252458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235709+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:01.235709+0000","last_peered":"2023-10-08T09:54:01.235709+0000","last_clean":"2023-10-08T09:54:01.235709+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:01.235709+0000","last_undegraded":"2023-10-08T09:54:01.235709+0000","last_fullsized":"2023-10-08T09:54:01.235709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4730,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":14704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":456335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.77100000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.60499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.879},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.85799999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1050739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78500000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.90800000000000003}]}]},{"osd":9,"up_from":63,"seq":270582939672,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":456917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.504}]},{"osd":2,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.623},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.374},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.39600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.79600000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.747},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72299999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":456939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85599999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.628}]},{"osd":6,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79700000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.93300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1531904,"data_stored":1047285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67400000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.49199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90800000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98399999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.628}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.75}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":456335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.26800000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.52100000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.878}]},{"osd":2,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.621,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60799999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.84099999999999997,"15min":0.876},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93000000000000005,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86099999999999999,"5min":0.876,"15min":0.876},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68100000000000005,"15min":0.68799999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.877,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.75600000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.83699999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.745},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.79500000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.99299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":460177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69499999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.83499999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.53700000000000003,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65800000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.438,"15min":0.42299999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.90000000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.625,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67800000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73799999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69999999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":456425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.46800000000000003},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52100000000000002,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84999999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":2.206,"15min":2.206},"last":0.443},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.38100000000000001},"min":{"1min":0.28199999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47399999999999998,"15min":0.46200000000000002},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.30299999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73999999999999999,"5min":1.04,"15min":1.04},"last":0.504},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67100000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81299999999999994,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.628},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.30099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.66300000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63100000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68600000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":460211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":2.698,"15min":2.698},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.501,"15min":0.47599999999999998},"min":{"1min":0.41899999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76200000000000001,"5min":1.024,"15min":1.024},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.48799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71999999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53900000000000003,"5min":0.621,"15min":0.621},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.28799999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57899999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.47099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.76400000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.46700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.95299999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.51700000000000002,"15min":0.501},"min":{"1min":0.38100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79700000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.98699999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.751},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.85099999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.873},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":456357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.31900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.41999999999999998,"15min":0.40100000000000002},"min":{"1min":0.33200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63100000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78700000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.42299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.75800000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.443,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.439,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.625,"15min":0.624},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81699999999999995,"5min":0.874,"15min":0.874},"last":0.71899999999999997}]},{"osd":6,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77800000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.88800000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.492,"5min":5.492,"15min":5.492},"last":0.874}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":5,"up_from":37,"seq":158913789992,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1046885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68300000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86599999999999999,"5min":1.379,"15min":1.379},"last":0.628}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":1.407,"15min":1.407},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":1.476,"15min":1.476},"last":0.55800000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.54000000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.72599999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98599999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60999999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.754},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:54:25.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:54:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 -- 172.21.15.97:0/2974484422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac101100 msgr2=0x7f96ac1079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 --2- 172.21.15.97:0/2974484422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac101100 0x7f96ac1079b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b0d0 tx=0x7f96a400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 -- 172.21.15.97:0/2974484422 shutdown_connections 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 --2- 172.21.15.97:0/2974484422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac101100 0x7f96ac1079b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 --2- 172.21.15.97:0/2974484422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac0ff570 0x7f96ac0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 --2- 172.21.15.97:0/2974484422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 0x7f96ac100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 -- 172.21.15.97:0/2974484422 >> 172.21.15.97:0/2974484422 conn(0x7f96ac0faae0 msgr2=0x7f96ac0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 -- 172.21.15.97:0/2974484422 shutdown_connections 2023-10-08T09:54:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.429+0000 7f96abfff700 1 -- 172.21.15.97:0/2974484422 wait complete. 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 Processor -- start 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 -- start start 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 0x7f96ac19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac101100 0x7f96ac19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1a0400 con 0x7f96ac0ff570 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1a0570 con 0x7f96ac101100 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96abfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1a0730 con 0x7f96ac100760 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96aaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96aaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55626/0 (socket says 172.21.15.97:55626) 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96aaffd700 1 -- 172.21.15.97:0/1536175266 learned_addr learned my addr 172.21.15.97:0/1536175266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96aa7fc700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 0x7f96ac19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.430+0000 7f96ab7fe700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac101100 0x7f96ac19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 -- 172.21.15.97:0/1536175266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 msgr2=0x7f96ac19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 0x7f96ac19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 -- 172.21.15.97:0/1536175266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac101100 msgr2=0x7f96ac19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac101100 0x7f96ac19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 -- 172.21.15.97:0/1536175266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4009d00 con 0x7f96ac0ff570 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96aaffd700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f969400d980 tx=0x7f969400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400de60 con 0x7f96ac0ff570 2023-10-08T09:54:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac1a09b0 con 0x7f96ac0ff570 2023-10-08T09:54:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96940044e0 con 0x7f96ac0ff570 2023-10-08T09:54:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694003be0 con 0x7f96ac0ff570 2023-10-08T09:54:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.431+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac1a0f10 con 0x7f96ac0ff570 2023-10-08T09:54:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.432+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac106920 con 0x7f96ac0ff570 2023-10-08T09:54:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.433+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9694009bb0 con 0x7f96ac0ff570 2023-10-08T09:54:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.434+0000 7f96b0a02700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 0x7f96980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.434+0000 7f96aa7fc700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 0x7f96980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.434+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f96940ccfe0 con 0x7f96ac0ff570 2023-10-08T09:54:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.435+0000 7f96aa7fc700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 0x7f96980a3940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96ac069ce0 tx=0x7f969c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.438+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9694096410 con 0x7f96ac0ff570 2023-10-08T09:54:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:54:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:active 2023-10-08T09:54:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T09:54:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:54:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2768852966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:25 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2882045827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:active 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2768852966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:25 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2882045827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/295723462' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] up:active 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2768852966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:25 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2882045827' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.627+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96ac060c30 con 0x7f96980a1480 2023-10-08T09:54:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.639+0000 7f96b0a02700 1 -- 172.21.15.97:0/1536175266 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423186 (secure 0 0 0) 0x7f96ac060c30 con 0x7f96980a1480 2023-10-08T09:54:25.645 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 msgr2=0x7f96980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 0x7f96980a3940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96ac069ce0 tx=0x7f969c009260 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 msgr2=0x7f96ac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f969400d980 tx=0x7f969400dd40 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 shutdown_connections 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f96980a1480 0x7f96980a3940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ac101100 0x7f96ac19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0ff570 0x7f96ac069740 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 --2- 172.21.15.97:0/1536175266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96ac100760 0x7f96ac19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.643+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 >> 172.21.15.97:0/1536175266 conn(0x7f96ac0faae0 msgr2=0x7f96ac103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.644+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 shutdown_connections 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:25.644+0000 7f96abfff700 1 -- 172.21.15.97:0/1536175266 wait complete. 2023-10-08T09:54:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:54:25.657 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-10-08T09:54:23.881110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595098,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":237,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337028,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604796,"statfs":{"total":1151940427776,"available":1151595311104,"internally_reserved":0,"allocated":13094912,"data_stored":7261224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":231,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236714+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:01.236714+0000","last_peered":"2023-10-08T09:54:01.236714+0000","last_clean":"2023-10-08T09:54:01.236714+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:01.236714+0000","last_undegraded":"2023-10-08T09:54:01.236714+0000","last_fullsized":"2023-10-08T09:54:01.236714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233407+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:01.233407+0000","last_peered":"2023-10-08T09:54:01.233407+0000","last_clean":"2023-10-08T09:54:01.233407+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:01.233407+0000","last_undegraded":"2023-10-08T09:54:01.233407+0000","last_fullsized":"2023-10-08T09:54:01.233407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234233+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:01.234233+0000","last_peered":"2023-10-08T09:54:01.234233+0000","last_clean":"2023-10-08T09:54:01.234233+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:01.234233+0000","last_undegraded":"2023-10-08T09:54:01.234233+0000","last_fullsized":"2023-10-08T09:54:01.234233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236452+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:01.236452+0000","last_peered":"2023-10-08T09:54:01.236452+0000","last_clean":"2023-10-08T09:54:01.236452+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:01.236452+0000","last_undegraded":"2023-10-08T09:54:01.236452+0000","last_fullsized":"2023-10-08T09:54:01.236452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227791+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:01.227791+0000","last_peered":"2023-10-08T09:54:01.227791+0000","last_clean":"2023-10-08T09:54:01.227791+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:01.227791+0000","last_undegraded":"2023-10-08T09:54:01.227791+0000","last_fullsized":"2023-10-08T09:54:01.227791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303490+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:01.303490+0000","last_peered":"2023-10-08T09:54:01.303490+0000","last_clean":"2023-10-08T09:54:01.303490+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:01.303490+0000","last_undegraded":"2023-10-08T09:54:01.303490+0000","last_fullsized":"2023-10-08T09:54:01.303490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234585+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:01.234585+0000","last_peered":"2023-10-08T09:54:01.234585+0000","last_clean":"2023-10-08T09:54:01.234585+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:01.234585+0000","last_undegraded":"2023-10-08T09:54:01.234585+0000","last_fullsized":"2023-10-08T09:54:01.234585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235130+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:01.235130+0000","last_peered":"2023-10-08T09:54:01.235130+0000","last_clean":"2023-10-08T09:54:01.235130+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:01.235130+0000","last_undegraded":"2023-10-08T09:54:01.235130+0000","last_fullsized":"2023-10-08T09:54:01.235130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228339+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:54:01.228339+0000","last_peered":"2023-10-08T09:54:01.228339+0000","last_clean":"2023-10-08T09:54:01.228339+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:54:01.228339+0000","last_undegraded":"2023-10-08T09:54:01.228339+0000","last_fullsized":"2023-10-08T09:54:01.228339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303360+0000","last_change":"2023-10-08T09:53:56.901387+0000","last_active":"2023-10-08T09:54:01.303360+0000","last_peered":"2023-10-08T09:54:01.303360+0000","last_clean":"2023-10-08T09:54:01.303360+0000","last_became_active":"2023-10-08T09:53:56.901091+0000","last_became_peered":"2023-10-08T09:53:56.901091+0000","last_unstale":"2023-10-08T09:54:01.303360+0000","last_undegraded":"2023-10-08T09:54:01.303360+0000","last_fullsized":"2023-10-08T09:54:01.303360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:01.441142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235763+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:01.235763+0000","last_peered":"2023-10-08T09:54:01.235763+0000","last_clean":"2023-10-08T09:54:01.235763+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:01.235763+0000","last_undegraded":"2023-10-08T09:54:01.235763+0000","last_fullsized":"2023-10-08T09:54:01.235763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236350+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:01.236350+0000","last_peered":"2023-10-08T09:54:01.236350+0000","last_clean":"2023-10-08T09:54:01.236350+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:01.236350+0000","last_undegraded":"2023-10-08T09:54:01.236350+0000","last_fullsized":"2023-10-08T09:54:01.236350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252372+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:01.252372+0000","last_peered":"2023-10-08T09:54:01.252372+0000","last_clean":"2023-10-08T09:54:01.252372+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:01.252372+0000","last_undegraded":"2023-10-08T09:54:01.252372+0000","last_fullsized":"2023-10-08T09:54:01.252372+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233417+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:01.233417+0000","last_peered":"2023-10-08T09:54:01.233417+0000","last_clean":"2023-10-08T09:54:01.233417+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:01.233417+0000","last_undegraded":"2023-10-08T09:54:01.233417+0000","last_fullsized":"2023-10-08T09:54:01.233417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233696+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:01.233696+0000","last_peered":"2023-10-08T09:54:01.233696+0000","last_clean":"2023-10-08T09:54:01.233696+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:01.233696+0000","last_undegraded":"2023-10-08T09:54:01.233696+0000","last_fullsized":"2023-10-08T09:54:01.233696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227867+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:54:01.227867+0000","last_peered":"2023-10-08T09:54:01.227867+0000","last_clean":"2023-10-08T09:54:01.227867+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:54:01.227867+0000","last_undegraded":"2023-10-08T09:54:01.227867+0000","last_fullsized":"2023-10-08T09:54:01.227867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236625+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:01.236625+0000","last_peered":"2023-10-08T09:54:01.236625+0000","last_clean":"2023-10-08T09:54:01.236625+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:01.236625+0000","last_undegraded":"2023-10-08T09:54:01.236625+0000","last_fullsized":"2023-10-08T09:54:01.236625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241845+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:01.241845+0000","last_peered":"2023-10-08T09:54:01.241845+0000","last_clean":"2023-10-08T09:54:01.241845+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:01.241845+0000","last_undegraded":"2023-10-08T09:54:01.241845+0000","last_fullsized":"2023-10-08T09:54:01.241845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234927+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:01.234927+0000","last_peered":"2023-10-08T09:54:01.234927+0000","last_clean":"2023-10-08T09:54:01.234927+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:01.234927+0000","last_undegraded":"2023-10-08T09:54:01.234927+0000","last_fullsized":"2023-10-08T09:54:01.234927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242614+0000","last_change":"2023-10-08T09:54:01.240656+0000","last_active":"2023-10-08T09:54:02.242614+0000","last_peered":"2023-10-08T09:54:02.242614+0000","last_clean":"2023-10-08T09:54:02.242614+0000","last_became_active":"2023-10-08T09:54:01.240274+0000","last_became_peered":"2023-10-08T09:54:01.240274+0000","last_unstale":"2023-10-08T09:54:02.242614+0000","last_undegraded":"2023-10-08T09:54:02.242614+0000","last_fullsized":"2023-10-08T09:54:02.242614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:14.800643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241838+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:01.241838+0000","last_peered":"2023-10-08T09:54:01.241838+0000","last_clean":"2023-10-08T09:54:01.241838+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:01.241838+0000","last_undegraded":"2023-10-08T09:54:01.241838+0000","last_fullsized":"2023-10-08T09:54:01.241838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241219+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:01.241219+0000","last_peered":"2023-10-08T09:54:01.241219+0000","last_clean":"2023-10-08T09:54:01.241219+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:01.241219+0000","last_undegraded":"2023-10-08T09:54:01.241219+0000","last_fullsized":"2023-10-08T09:54:01.241219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502709+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:03.502709+0000","last_peered":"2023-10-08T09:54:03.502709+0000","last_clean":"2023-10-08T09:54:03.502709+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:03.502709+0000","last_undegraded":"2023-10-08T09:54:03.502709+0000","last_fullsized":"2023-10-08T09:54:03.502709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401922+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:02.401922+0000","last_peered":"2023-10-08T09:54:02.401922+0000","last_clean":"2023-10-08T09:54:02.401922+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:02.401922+0000","last_undegraded":"2023-10-08T09:54:02.401922+0000","last_fullsized":"2023-10-08T09:54:02.401922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233799+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:01.233799+0000","last_peered":"2023-10-08T09:54:01.233799+0000","last_clean":"2023-10-08T09:54:01.233799+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:01.233799+0000","last_undegraded":"2023-10-08T09:54:01.233799+0000","last_fullsized":"2023-10-08T09:54:01.233799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502990+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:03.502990+0000","last_peered":"2023-10-08T09:54:03.502990+0000","last_clean":"2023-10-08T09:54:03.502990+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:03.502990+0000","last_undegraded":"2023-10-08T09:54:03.502990+0000","last_fullsized":"2023-10-08T09:54:03.502990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228305+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:01.228305+0000","last_peered":"2023-10-08T09:54:01.228305+0000","last_clean":"2023-10-08T09:54:01.228305+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:01.228305+0000","last_undegraded":"2023-10-08T09:54:01.228305+0000","last_fullsized":"2023-10-08T09:54:01.228305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228030+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:54:01.228030+0000","last_peered":"2023-10-08T09:54:01.228030+0000","last_clean":"2023-10-08T09:54:01.228030+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:54:01.228030+0000","last_undegraded":"2023-10-08T09:54:01.228030+0000","last_fullsized":"2023-10-08T09:54:01.228030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502747+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:03.502747+0000","last_peered":"2023-10-08T09:54:03.502747+0000","last_clean":"2023-10-08T09:54:03.502747+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:03.502747+0000","last_undegraded":"2023-10-08T09:54:03.502747+0000","last_fullsized":"2023-10-08T09:54:03.502747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228285+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:01.228285+0000","last_peered":"2023-10-08T09:54:01.228285+0000","last_clean":"2023-10-08T09:54:01.228285+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:01.228285+0000","last_undegraded":"2023-10-08T09:54:01.228285+0000","last_fullsized":"2023-10-08T09:54:01.228285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230787+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:01.230787+0000","last_peered":"2023-10-08T09:54:01.230787+0000","last_clean":"2023-10-08T09:54:01.230787+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:01.230787+0000","last_undegraded":"2023-10-08T09:54:01.230787+0000","last_fullsized":"2023-10-08T09:54:01.230787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243324+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:02.243324+0000","last_peered":"2023-10-08T09:54:02.243324+0000","last_clean":"2023-10-08T09:54:02.243324+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:02.243324+0000","last_undegraded":"2023-10-08T09:54:02.243324+0000","last_fullsized":"2023-10-08T09:54:02.243324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230400+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:54:01.230400+0000","last_peered":"2023-10-08T09:54:01.230400+0000","last_clean":"2023-10-08T09:54:01.230400+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:54:01.230400+0000","last_undegraded":"2023-10-08T09:54:01.230400+0000","last_fullsized":"2023-10-08T09:54:01.230400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228580+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:01.228580+0000","last_peered":"2023-10-08T09:54:01.228580+0000","last_clean":"2023-10-08T09:54:01.228580+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:01.228580+0000","last_undegraded":"2023-10-08T09:54:01.228580+0000","last_fullsized":"2023-10-08T09:54:01.228580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243923+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:02.243923+0000","last_peered":"2023-10-08T09:54:02.243923+0000","last_clean":"2023-10-08T09:54:02.243923+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:02.243923+0000","last_undegraded":"2023-10-08T09:54:02.243923+0000","last_fullsized":"2023-10-08T09:54:02.243923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.302998+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:01.302998+0000","last_peered":"2023-10-08T09:54:01.302998+0000","last_clean":"2023-10-08T09:54:01.302998+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:01.302998+0000","last_undegraded":"2023-10-08T09:54:01.302998+0000","last_fullsized":"2023-10-08T09:54:01.302998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230447+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:01.230447+0000","last_peered":"2023-10-08T09:54:01.230447+0000","last_clean":"2023-10-08T09:54:01.230447+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:01.230447+0000","last_undegraded":"2023-10-08T09:54:01.230447+0000","last_fullsized":"2023-10-08T09:54:01.230447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502863+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:03.502863+0000","last_peered":"2023-10-08T09:54:03.502863+0000","last_clean":"2023-10-08T09:54:03.502863+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:03.502863+0000","last_undegraded":"2023-10-08T09:54:03.502863+0000","last_fullsized":"2023-10-08T09:54:03.502863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252294+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:01.252294+0000","last_peered":"2023-10-08T09:54:01.252294+0000","last_clean":"2023-10-08T09:54:01.252294+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:01.252294+0000","last_undegraded":"2023-10-08T09:54:01.252294+0000","last_fullsized":"2023-10-08T09:54:01.252294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235096+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:01.235096+0000","last_peered":"2023-10-08T09:54:01.235096+0000","last_clean":"2023-10-08T09:54:01.235096+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:01.235096+0000","last_undegraded":"2023-10-08T09:54:01.235096+0000","last_fullsized":"2023-10-08T09:54:01.235096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502710+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:03.502710+0000","last_peered":"2023-10-08T09:54:03.502710+0000","last_clean":"2023-10-08T09:54:03.502710+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:03.502710+0000","last_undegraded":"2023-10-08T09:54:03.502710+0000","last_fullsized":"2023-10-08T09:54:03.502710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230271+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:01.230271+0000","last_peered":"2023-10-08T09:54:01.230271+0000","last_clean":"2023-10-08T09:54:01.230271+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:01.230271+0000","last_undegraded":"2023-10-08T09:54:01.230271+0000","last_fullsized":"2023-10-08T09:54:01.230271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240133+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:01.240133+0000","last_peered":"2023-10-08T09:54:01.240133+0000","last_clean":"2023-10-08T09:54:01.240133+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:01.240133+0000","last_undegraded":"2023-10-08T09:54:01.240133+0000","last_fullsized":"2023-10-08T09:54:01.240133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401534+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:02.401534+0000","last_peered":"2023-10-08T09:54:02.401534+0000","last_clean":"2023-10-08T09:54:02.401534+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:02.401534+0000","last_undegraded":"2023-10-08T09:54:02.401534+0000","last_fullsized":"2023-10-08T09:54:02.401534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303177+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:54:01.303177+0000","last_peered":"2023-10-08T09:54:01.303177+0000","last_clean":"2023-10-08T09:54:01.303177+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:54:01.303177+0000","last_undegraded":"2023-10-08T09:54:01.303177+0000","last_fullsized":"2023-10-08T09:54:01.303177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228327+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:54:01.228327+0000","last_peered":"2023-10-08T09:54:01.228327+0000","last_clean":"2023-10-08T09:54:01.228327+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:54:01.228327+0000","last_undegraded":"2023-10-08T09:54:01.228327+0000","last_fullsized":"2023-10-08T09:54:01.228327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.501874+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:03.501874+0000","last_peered":"2023-10-08T09:54:03.501874+0000","last_clean":"2023-10-08T09:54:03.501874+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:03.501874+0000","last_undegraded":"2023-10-08T09:54:03.501874+0000","last_fullsized":"2023-10-08T09:54:03.501874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235442+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:01.235442+0000","last_peered":"2023-10-08T09:54:01.235442+0000","last_clean":"2023-10-08T09:54:01.235442+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:01.235442+0000","last_undegraded":"2023-10-08T09:54:01.235442+0000","last_fullsized":"2023-10-08T09:54:01.235442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233693+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:01.233693+0000","last_peered":"2023-10-08T09:54:01.233693+0000","last_clean":"2023-10-08T09:54:01.233693+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:01.233693+0000","last_undegraded":"2023-10-08T09:54:01.233693+0000","last_fullsized":"2023-10-08T09:54:01.233693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503155+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:03.503155+0000","last_peered":"2023-10-08T09:54:03.503155+0000","last_clean":"2023-10-08T09:54:03.503155+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:03.503155+0000","last_undegraded":"2023-10-08T09:54:03.503155+0000","last_fullsized":"2023-10-08T09:54:03.503155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230618+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:01.230618+0000","last_peered":"2023-10-08T09:54:01.230618+0000","last_clean":"2023-10-08T09:54:01.230618+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:01.230618+0000","last_undegraded":"2023-10-08T09:54:01.230618+0000","last_fullsized":"2023-10-08T09:54:01.230618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228133+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:54:01.228133+0000","last_peered":"2023-10-08T09:54:01.228133+0000","last_clean":"2023-10-08T09:54:01.228133+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:54:01.228133+0000","last_undegraded":"2023-10-08T09:54:01.228133+0000","last_fullsized":"2023-10-08T09:54:01.228133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502416+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:03.502416+0000","last_peered":"2023-10-08T09:54:03.502416+0000","last_clean":"2023-10-08T09:54:03.502416+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:03.502416+0000","last_undegraded":"2023-10-08T09:54:03.502416+0000","last_fullsized":"2023-10-08T09:54:03.502416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'10","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:13.023680+0000","last_change":"2023-10-08T09:53:56.203103+0000","last_active":"2023-10-08T09:54:13.023680+0000","last_peered":"2023-10-08T09:54:13.023680+0000","last_clean":"2023-10-08T09:54:13.023680+0000","last_became_active":"2023-10-08T09:53:56.202807+0000","last_became_peered":"2023-10-08T09:53:56.202807+0000","last_unstale":"2023-10-08T09:54:13.023680+0000","last_undegraded":"2023-10-08T09:54:13.023680+0000","last_fullsized":"2023-10-08T09:54:13.023680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:01.154899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228142+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:01.228142+0000","last_peered":"2023-10-08T09:54:01.228142+0000","last_clean":"2023-10-08T09:54:01.228142+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:01.228142+0000","last_undegraded":"2023-10-08T09:54:01.228142+0000","last_fullsized":"2023-10-08T09:54:01.228142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243461+0000","last_change":"2023-10-08T09:54:01.251015+0000","last_active":"2023-10-08T09:54:02.243461+0000","last_peered":"2023-10-08T09:54:02.243461+0000","last_clean":"2023-10-08T09:54:02.243461+0000","last_became_active":"2023-10-08T09:54:01.250599+0000","last_became_peered":"2023-10-08T09:54:01.250599+0000","last_unstale":"2023-10-08T09:54:02.243461+0000","last_undegraded":"2023-10-08T09:54:02.243461+0000","last_fullsized":"2023-10-08T09:54:02.243461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:59.645562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234184+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:01.234184+0000","last_peered":"2023-10-08T09:54:01.234184+0000","last_clean":"2023-10-08T09:54:01.234184+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:01.234184+0000","last_undegraded":"2023-10-08T09:54:01.234184+0000","last_fullsized":"2023-10-08T09:54:01.234184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252351+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:01.252351+0000","last_peered":"2023-10-08T09:54:01.252351+0000","last_clean":"2023-10-08T09:54:01.252351+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:01.252351+0000","last_undegraded":"2023-10-08T09:54:01.252351+0000","last_fullsized":"2023-10-08T09:54:01.252351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502668+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:03.502668+0000","last_peered":"2023-10-08T09:54:03.502668+0000","last_clean":"2023-10-08T09:54:03.502668+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:03.502668+0000","last_undegraded":"2023-10-08T09:54:03.502668+0000","last_fullsized":"2023-10-08T09:54:03.502668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230348+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:01.230348+0000","last_peered":"2023-10-08T09:54:01.230348+0000","last_clean":"2023-10-08T09:54:01.230348+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:01.230348+0000","last_undegraded":"2023-10-08T09:54:01.230348+0000","last_fullsized":"2023-10-08T09:54:01.230348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.236418+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:01.236418+0000","last_peered":"2023-10-08T09:54:01.236418+0000","last_clean":"2023-10-08T09:54:01.236418+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:01.236418+0000","last_undegraded":"2023-10-08T09:54:01.236418+0000","last_fullsized":"2023-10-08T09:54:01.236418+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502141+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:03.502141+0000","last_peered":"2023-10-08T09:54:03.502141+0000","last_clean":"2023-10-08T09:54:03.502141+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:03.502141+0000","last_undegraded":"2023-10-08T09:54:03.502141+0000","last_fullsized":"2023-10-08T09:54:03.502141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.232857+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:54:01.232857+0000","last_peered":"2023-10-08T09:54:01.232857+0000","last_clean":"2023-10-08T09:54:01.232857+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:54:01.232857+0000","last_undegraded":"2023-10-08T09:54:01.232857+0000","last_fullsized":"2023-10-08T09:54:01.232857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234983+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:01.234983+0000","last_peered":"2023-10-08T09:54:01.234983+0000","last_clean":"2023-10-08T09:54:01.234983+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:01.234983+0000","last_undegraded":"2023-10-08T09:54:01.234983+0000","last_fullsized":"2023-10-08T09:54:01.234983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.278175+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:02.278175+0000","last_peered":"2023-10-08T09:54:02.278175+0000","last_clean":"2023-10-08T09:54:02.278175+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:02.278175+0000","last_undegraded":"2023-10-08T09:54:02.278175+0000","last_fullsized":"2023-10-08T09:54:02.278175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227722+0000","last_change":"2023-10-08T09:53:56.200805+0000","last_active":"2023-10-08T09:54:01.227722+0000","last_peered":"2023-10-08T09:54:01.227722+0000","last_clean":"2023-10-08T09:54:01.227722+0000","last_became_active":"2023-10-08T09:53:56.200507+0000","last_became_peered":"2023-10-08T09:53:56.200507+0000","last_unstale":"2023-10-08T09:54:01.227722+0000","last_undegraded":"2023-10-08T09:54:01.227722+0000","last_fullsized":"2023-10-08T09:54:01.227722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.328356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230299+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:01.230299+0000","last_peered":"2023-10-08T09:54:01.230299+0000","last_clean":"2023-10-08T09:54:01.230299+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:01.230299+0000","last_undegraded":"2023-10-08T09:54:01.230299+0000","last_fullsized":"2023-10-08T09:54:01.230299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242768+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:02.242768+0000","last_peered":"2023-10-08T09:54:02.242768+0000","last_clean":"2023-10-08T09:54:02.242768+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:02.242768+0000","last_undegraded":"2023-10-08T09:54:02.242768+0000","last_fullsized":"2023-10-08T09:54:02.242768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228124+0000","last_change":"2023-10-08T09:53:56.199894+0000","last_active":"2023-10-08T09:54:01.228124+0000","last_peered":"2023-10-08T09:54:01.228124+0000","last_clean":"2023-10-08T09:54:01.228124+0000","last_became_active":"2023-10-08T09:53:56.199576+0000","last_became_peered":"2023-10-08T09:53:56.199576+0000","last_unstale":"2023-10-08T09:54:01.228124+0000","last_undegraded":"2023-10-08T09:54:01.228124+0000","last_fullsized":"2023-10-08T09:54:01.228124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:16.389646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235084+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:01.235084+0000","last_peered":"2023-10-08T09:54:01.235084+0000","last_clean":"2023-10-08T09:54:01.235084+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:01.235084+0000","last_undegraded":"2023-10-08T09:54:01.235084+0000","last_fullsized":"2023-10-08T09:54:01.235084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.590187+0000","last_change":"2023-10-08T09:54:01.304336+0000","last_active":"2023-10-08T09:54:03.590187+0000","last_peered":"2023-10-08T09:54:03.590187+0000","last_clean":"2023-10-08T09:54:03.590187+0000","last_became_active":"2023-10-08T09:54:01.303870+0000","last_became_peered":"2023-10-08T09:54:01.303870+0000","last_unstale":"2023-10-08T09:54:03.590187+0000","last_undegraded":"2023-10-08T09:54:03.590187+0000","last_fullsized":"2023-10-08T09:54:03.590187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:24.098993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240484+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:01.240484+0000","last_peered":"2023-10-08T09:54:01.240484+0000","last_clean":"2023-10-08T09:54:01.240484+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:01.240484+0000","last_undegraded":"2023-10-08T09:54:01.240484+0000","last_fullsized":"2023-10-08T09:54:01.240484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.303050+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:01.303050+0000","last_peered":"2023-10-08T09:54:01.303050+0000","last_clean":"2023-10-08T09:54:01.303050+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:01.303050+0000","last_undegraded":"2023-10-08T09:54:01.303050+0000","last_fullsized":"2023-10-08T09:54:01.303050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.278006+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:02.278006+0000","last_peered":"2023-10-08T09:54:02.278006+0000","last_clean":"2023-10-08T09:54:02.278006+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:02.278006+0000","last_undegraded":"2023-10-08T09:54:02.278006+0000","last_fullsized":"2023-10-08T09:54:02.278006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234808+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:01.234808+0000","last_peered":"2023-10-08T09:54:01.234808+0000","last_clean":"2023-10-08T09:54:01.234808+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:01.234808+0000","last_undegraded":"2023-10-08T09:54:01.234808+0000","last_fullsized":"2023-10-08T09:54:01.234808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.302843+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:01.302843+0000","last_peered":"2023-10-08T09:54:01.302843+0000","last_clean":"2023-10-08T09:54:01.302843+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:01.302843+0000","last_undegraded":"2023-10-08T09:54:01.302843+0000","last_fullsized":"2023-10-08T09:54:01.302843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.277886+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:02.277886+0000","last_peered":"2023-10-08T09:54:02.277886+0000","last_clean":"2023-10-08T09:54:02.277886+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:02.277886+0000","last_undegraded":"2023-10-08T09:54:02.277886+0000","last_fullsized":"2023-10-08T09:54:02.277886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230726+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:01.230726+0000","last_peered":"2023-10-08T09:54:01.230726+0000","last_clean":"2023-10-08T09:54:01.230726+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:01.230726+0000","last_undegraded":"2023-10-08T09:54:01.230726+0000","last_fullsized":"2023-10-08T09:54:01.230726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252120+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:54:01.252120+0000","last_peered":"2023-10-08T09:54:01.252120+0000","last_clean":"2023-10-08T09:54:01.252120+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:54:01.252120+0000","last_undegraded":"2023-10-08T09:54:01.252120+0000","last_fullsized":"2023-10-08T09:54:01.252120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503294+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:03.503294+0000","last_peered":"2023-10-08T09:54:03.503294+0000","last_clean":"2023-10-08T09:54:03.503294+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:03.503294+0000","last_undegraded":"2023-10-08T09:54:03.503294+0000","last_fullsized":"2023-10-08T09:54:03.503294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240585+0000","last_change":"2023-10-08T09:53:56.887772+0000","last_active":"2023-10-08T09:54:01.240585+0000","last_peered":"2023-10-08T09:54:01.240585+0000","last_clean":"2023-10-08T09:54:01.240585+0000","last_became_active":"2023-10-08T09:53:56.887155+0000","last_became_peered":"2023-10-08T09:53:56.887155+0000","last_unstale":"2023-10-08T09:54:01.240585+0000","last_undegraded":"2023-10-08T09:54:01.240585+0000","last_fullsized":"2023-10-08T09:54:01.240585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:45.184494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234865+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:01.234865+0000","last_peered":"2023-10-08T09:54:01.234865+0000","last_clean":"2023-10-08T09:54:01.234865+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:01.234865+0000","last_undegraded":"2023-10-08T09:54:01.234865+0000","last_fullsized":"2023-10-08T09:54:01.234865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234778+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:01.234778+0000","last_peered":"2023-10-08T09:54:01.234778+0000","last_clean":"2023-10-08T09:54:01.234778+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:01.234778+0000","last_undegraded":"2023-10-08T09:54:01.234778+0000","last_fullsized":"2023-10-08T09:54:01.234778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243192+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:02.243192+0000","last_peered":"2023-10-08T09:54:02.243192+0000","last_clean":"2023-10-08T09:54:02.243192+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:02.243192+0000","last_undegraded":"2023-10-08T09:54:02.243192+0000","last_fullsized":"2023-10-08T09:54:02.243192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252114+0000","last_change":"2023-10-08T09:53:56.897985+0000","last_active":"2023-10-08T09:54:01.252114+0000","last_peered":"2023-10-08T09:54:01.252114+0000","last_clean":"2023-10-08T09:54:01.252114+0000","last_became_active":"2023-10-08T09:53:56.897520+0000","last_became_peered":"2023-10-08T09:53:56.897520+0000","last_unstale":"2023-10-08T09:54:01.252114+0000","last_undegraded":"2023-10-08T09:54:01.252114+0000","last_fullsized":"2023-10-08T09:54:01.252114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:52.733642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235448+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:01.235448+0000","last_peered":"2023-10-08T09:54:01.235448+0000","last_clean":"2023-10-08T09:54:01.235448+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:01.235448+0000","last_undegraded":"2023-10-08T09:54:01.235448+0000","last_fullsized":"2023-10-08T09:54:01.235448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502738+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:03.502738+0000","last_peered":"2023-10-08T09:54:03.502738+0000","last_clean":"2023-10-08T09:54:03.502738+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:03.502738+0000","last_undegraded":"2023-10-08T09:54:03.502738+0000","last_fullsized":"2023-10-08T09:54:03.502738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228431+0000","last_change":"2023-10-08T09:53:56.195815+0000","last_active":"2023-10-08T09:54:01.228431+0000","last_peered":"2023-10-08T09:54:01.228431+0000","last_clean":"2023-10-08T09:54:01.228431+0000","last_became_active":"2023-10-08T09:53:56.195472+0000","last_became_peered":"2023-10-08T09:53:56.195472+0000","last_unstale":"2023-10-08T09:54:01.228431+0000","last_undegraded":"2023-10-08T09:54:01.228431+0000","last_fullsized":"2023-10-08T09:54:01.228431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:53.341794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252674+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:01.252674+0000","last_peered":"2023-10-08T09:54:01.252674+0000","last_clean":"2023-10-08T09:54:01.252674+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:01.252674+0000","last_undegraded":"2023-10-08T09:54:01.252674+0000","last_fullsized":"2023-10-08T09:54:01.252674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.401282+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:02.401282+0000","last_peered":"2023-10-08T09:54:02.401282+0000","last_clean":"2023-10-08T09:54:02.401282+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:02.401282+0000","last_undegraded":"2023-10-08T09:54:02.401282+0000","last_fullsized":"2023-10-08T09:54:02.401282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241933+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:01.241933+0000","last_peered":"2023-10-08T09:54:01.241933+0000","last_clean":"2023-10-08T09:54:01.241933+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:01.241933+0000","last_undegraded":"2023-10-08T09:54:01.241933+0000","last_fullsized":"2023-10-08T09:54:01.241933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240985+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:01.240985+0000","last_peered":"2023-10-08T09:54:01.240985+0000","last_clean":"2023-10-08T09:54:01.240985+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:01.240985+0000","last_undegraded":"2023-10-08T09:54:01.240985+0000","last_fullsized":"2023-10-08T09:54:01.240985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.242332+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:02.242332+0000","last_peered":"2023-10-08T09:54:02.242332+0000","last_clean":"2023-10-08T09:54:02.242332+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:02.242332+0000","last_undegraded":"2023-10-08T09:54:02.242332+0000","last_fullsized":"2023-10-08T09:54:02.242332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233031+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:01.233031+0000","last_peered":"2023-10-08T09:54:01.233031+0000","last_clean":"2023-10-08T09:54:01.233031+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:01.233031+0000","last_undegraded":"2023-10-08T09:54:01.233031+0000","last_fullsized":"2023-10-08T09:54:01.233031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233493+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:01.233493+0000","last_peered":"2023-10-08T09:54:01.233493+0000","last_clean":"2023-10-08T09:54:01.233493+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:01.233493+0000","last_undegraded":"2023-10-08T09:54:01.233493+0000","last_fullsized":"2023-10-08T09:54:01.233493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503322+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:03.503322+0000","last_peered":"2023-10-08T09:54:03.503322+0000","last_clean":"2023-10-08T09:54:03.503322+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:03.503322+0000","last_undegraded":"2023-10-08T09:54:03.503322+0000","last_fullsized":"2023-10-08T09:54:03.503322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:22.181296+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:54:22.181296+0000","last_peered":"2023-10-08T09:54:22.181296+0000","last_clean":"2023-10-08T09:54:22.181296+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:54:22.181296+0000","last_undegraded":"2023-10-08T09:54:22.181296+0000","last_fullsized":"2023-10-08T09:54:22.181296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227731+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:01.227731+0000","last_peered":"2023-10-08T09:54:01.227731+0000","last_clean":"2023-10-08T09:54:01.227731+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:01.227731+0000","last_undegraded":"2023-10-08T09:54:01.227731+0000","last_fullsized":"2023-10-08T09:54:01.227731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.502366+0000","last_change":"2023-10-08T09:54:01.246246+0000","last_active":"2023-10-08T09:54:03.502366+0000","last_peered":"2023-10-08T09:54:03.502366+0000","last_clean":"2023-10-08T09:54:03.502366+0000","last_became_active":"2023-10-08T09:54:01.245664+0000","last_became_peered":"2023-10-08T09:54:01.245664+0000","last_unstale":"2023-10-08T09:54:03.502366+0000","last_undegraded":"2023-10-08T09:54:03.502366+0000","last_fullsized":"2023-10-08T09:54:03.502366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:07.600486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:22.179798+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:22.179798+0000","last_peered":"2023-10-08T09:54:22.179798+0000","last_clean":"2023-10-08T09:54:22.179798+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:22.179798+0000","last_undegraded":"2023-10-08T09:54:22.179798+0000","last_fullsized":"2023-10-08T09:54:22.179798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234272+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:01.234272+0000","last_peered":"2023-10-08T09:54:01.234272+0000","last_clean":"2023-10-08T09:54:01.234272+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:01.234272+0000","last_undegraded":"2023-10-08T09:54:01.234272+0000","last_fullsized":"2023-10-08T09:54:01.234272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503449+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:03.503449+0000","last_peered":"2023-10-08T09:54:03.503449+0000","last_clean":"2023-10-08T09:54:03.503449+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:03.503449+0000","last_undegraded":"2023-10-08T09:54:03.503449+0000","last_fullsized":"2023-10-08T09:54:03.503449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233991+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:01.233991+0000","last_peered":"2023-10-08T09:54:01.233991+0000","last_clean":"2023-10-08T09:54:01.233991+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:01.233991+0000","last_undegraded":"2023-10-08T09:54:01.233991+0000","last_fullsized":"2023-10-08T09:54:01.233991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.240781+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:01.240781+0000","last_peered":"2023-10-08T09:54:01.240781+0000","last_clean":"2023-10-08T09:54:01.240781+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:01.240781+0000","last_undegraded":"2023-10-08T09:54:01.240781+0000","last_fullsized":"2023-10-08T09:54:01.240781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.503718+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:03.503718+0000","last_peered":"2023-10-08T09:54:03.503718+0000","last_clean":"2023-10-08T09:54:03.503718+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:03.503718+0000","last_undegraded":"2023-10-08T09:54:03.503718+0000","last_fullsized":"2023-10-08T09:54:03.503718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.227921+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:01.227921+0000","last_peered":"2023-10-08T09:54:01.227921+0000","last_clean":"2023-10-08T09:54:01.227921+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:01.227921+0000","last_undegraded":"2023-10-08T09:54:01.227921+0000","last_fullsized":"2023-10-08T09:54:01.227921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234986+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:01.234986+0000","last_peered":"2023-10-08T09:54:01.234986+0000","last_clean":"2023-10-08T09:54:01.234986+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:01.234986+0000","last_undegraded":"2023-10-08T09:54:01.234986+0000","last_fullsized":"2023-10-08T09:54:01.234986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.624263+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:54:02.624263+0000","last_peered":"2023-10-08T09:54:02.624263+0000","last_clean":"2023-10-08T09:54:02.624263+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:54:02.624263+0000","last_undegraded":"2023-10-08T09:54:02.624263+0000","last_fullsized":"2023-10-08T09:54:02.624263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234654+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:01.234654+0000","last_peered":"2023-10-08T09:54:01.234654+0000","last_clean":"2023-10-08T09:54:01.234654+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:01.234654+0000","last_undegraded":"2023-10-08T09:54:01.234654+0000","last_fullsized":"2023-10-08T09:54:01.234654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235314+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:01.235314+0000","last_peered":"2023-10-08T09:54:01.235314+0000","last_clean":"2023-10-08T09:54:01.235314+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:01.235314+0000","last_undegraded":"2023-10-08T09:54:01.235314+0000","last_fullsized":"2023-10-08T09:54:01.235314+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:03.501786+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:03.501786+0000","last_peered":"2023-10-08T09:54:03.501786+0000","last_clean":"2023-10-08T09:54:03.501786+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:03.501786+0000","last_undegraded":"2023-10-08T09:54:03.501786+0000","last_fullsized":"2023-10-08T09:54:03.501786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230841+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:01.230841+0000","last_peered":"2023-10-08T09:54:01.230841+0000","last_clean":"2023-10-08T09:54:01.230841+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:01.230841+0000","last_undegraded":"2023-10-08T09:54:01.230841+0000","last_fullsized":"2023-10-08T09:54:01.230841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235109+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:01.235109+0000","last_peered":"2023-10-08T09:54:01.235109+0000","last_clean":"2023-10-08T09:54:01.235109+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:01.235109+0000","last_undegraded":"2023-10-08T09:54:01.235109+0000","last_fullsized":"2023-10-08T09:54:01.235109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:54:02.243323+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:02.243323+0000","last_peered":"2023-10-08T09:54:02.243323+0000","last_clean":"2023-10-08T09:54:02.243323+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:02.243323+0000","last_undegraded":"2023-10-08T09:54:02.243323+0000","last_fullsized":"2023-10-08T09:54:02.243323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252739+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:01.252739+0000","last_peered":"2023-10-08T09:54:01.252739+0000","last_clean":"2023-10-08T09:54:01.252739+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:01.252739+0000","last_undegraded":"2023-10-08T09:54:01.252739+0000","last_fullsized":"2023-10-08T09:54:01.252739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235451+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:01.235451+0000","last_peered":"2023-10-08T09:54:01.235451+0000","last_clean":"2023-10-08T09:54:01.235451+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:01.235451+0000","last_undegraded":"2023-10-08T09:54:01.235451+0000","last_fullsized":"2023-10-08T09:54:01.235451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234365+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:01.234365+0000","last_peered":"2023-10-08T09:54:01.234365+0000","last_clean":"2023-10-08T09:54:01.234365+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:01.234365+0000","last_undegraded":"2023-10-08T09:54:01.234365+0000","last_fullsized":"2023-10-08T09:54:01.234365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234047+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:01.234047+0000","last_peered":"2023-10-08T09:54:01.234047+0000","last_clean":"2023-10-08T09:54:01.234047+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:01.234047+0000","last_undegraded":"2023-10-08T09:54:01.234047+0000","last_fullsized":"2023-10-08T09:54:01.234047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.233886+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:01.233886+0000","last_peered":"2023-10-08T09:54:01.233886+0000","last_clean":"2023-10-08T09:54:01.233886+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:01.233886+0000","last_undegraded":"2023-10-08T09:54:01.233886+0000","last_fullsized":"2023-10-08T09:54:01.233886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.234060+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:01.234060+0000","last_peered":"2023-10-08T09:54:01.234060+0000","last_clean":"2023-10-08T09:54:01.234060+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:01.234060+0000","last_undegraded":"2023-10-08T09:54:01.234060+0000","last_fullsized":"2023-10-08T09:54:01.234060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252584+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:01.252584+0000","last_peered":"2023-10-08T09:54:01.252584+0000","last_clean":"2023-10-08T09:54:01.252584+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:01.252584+0000","last_undegraded":"2023-10-08T09:54:01.252584+0000","last_fullsized":"2023-10-08T09:54:01.252584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235741+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:01.235741+0000","last_peered":"2023-10-08T09:54:01.235741+0000","last_clean":"2023-10-08T09:54:01.235741+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:01.235741+0000","last_undegraded":"2023-10-08T09:54:01.235741+0000","last_fullsized":"2023-10-08T09:54:01.235741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235051+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:01.235051+0000","last_peered":"2023-10-08T09:54:01.235051+0000","last_clean":"2023-10-08T09:54:01.235051+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:01.235051+0000","last_undegraded":"2023-10-08T09:54:01.235051+0000","last_fullsized":"2023-10-08T09:54:01.235051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.241060+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:01.241060+0000","last_peered":"2023-10-08T09:54:01.241060+0000","last_clean":"2023-10-08T09:54:01.241060+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:01.241060+0000","last_undegraded":"2023-10-08T09:54:01.241060+0000","last_fullsized":"2023-10-08T09:54:01.241060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.230165+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:01.230165+0000","last_peered":"2023-10-08T09:54:01.230165+0000","last_clean":"2023-10-08T09:54:01.230165+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:01.230165+0000","last_undegraded":"2023-10-08T09:54:01.230165+0000","last_fullsized":"2023-10-08T09:54:01.230165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.239394+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:01.239394+0000","last_peered":"2023-10-08T09:54:01.239394+0000","last_clean":"2023-10-08T09:54:01.239394+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:01.239394+0000","last_undegraded":"2023-10-08T09:54:01.239394+0000","last_fullsized":"2023-10-08T09:54:01.239394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.228627+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:54:01.228627+0000","last_peered":"2023-10-08T09:54:01.228627+0000","last_clean":"2023-10-08T09:54:01.228627+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:54:01.228627+0000","last_undegraded":"2023-10-08T09:54:01.228627+0000","last_fullsized":"2023-10-08T09:54:01.228627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.252458+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:01.252458+0000","last_peered":"2023-10-08T09:54:01.252458+0000","last_clean":"2023-10-08T09:54:01.252458+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:01.252458+0000","last_undegraded":"2023-10-08T09:54:01.252458+0000","last_fullsized":"2023-10-08T09:54:01.252458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-08T09:54:01.235709+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:01.235709+0000","last_peered":"2023-10-08T09:54:01.235709+0000","last_clean":"2023-10-08T09:54:01.235709+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:01.235709+0000","last_undegraded":"2023-10-08T09:54:01.235709+0000","last_fullsized":"2023-10-08T09:54:01.235709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4730,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":14704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":456335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.77100000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.60499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.879},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.85799999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1050739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78500000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.90800000000000003}]}]},{"osd":9,"up_from":63,"seq":270582939672,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":456917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.504}]},{"osd":2,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.623},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.374},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.39600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.79600000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.747},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72299999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":456939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67100000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85599999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.628}]},{"osd":6,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79700000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.93300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1531904,"data_stored":1047285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67400000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.49199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90800000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98399999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.628}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.75}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":6,"up_from":44,"seq":188978561060,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":456335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.26800000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.52100000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.878}]},{"osd":2,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.621,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60799999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.84099999999999997,"15min":0.876},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93000000000000005,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86099999999999999,"5min":0.876,"15min":0.876},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68100000000000005,"15min":0.68799999999999994},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.877,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.75600000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.83699999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.745},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.79500000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.99299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":460177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69499999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.83499999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.53700000000000003,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65800000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.438,"15min":0.42299999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.90000000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.625,"15min":0.626},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67800000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73799999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:53:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69999999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":456425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51300000000000001,"15min":0.46800000000000003},"min":{"1min":0.377,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73099999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52100000000000002,"15min":0.47599999999999998},"min":{"1min":0.38,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84999999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":2.206,"15min":2.206},"last":0.443},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.38700000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.38100000000000001},"min":{"1min":0.28199999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47399999999999998,"15min":0.46200000000000002},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.30299999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73999999999999999,"5min":1.04,"15min":1.04},"last":0.504},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67100000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81299999999999994,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.628},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72899999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.30099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":0.66300000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63100000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:53:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68600000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":460211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":2.698,"15min":2.698},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.501,"15min":0.47599999999999998},"min":{"1min":0.41899999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51300000000000001,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76200000000000001,"5min":1.024,"15min":1.024},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44700000000000001,"15min":0.439},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.48799999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71999999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:53:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53900000000000003,"5min":0.621,"15min":0.621},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.28799999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57899999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.47099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.76400000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:53:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80500000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.46700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.95299999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:53:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.71899999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.51700000000000002,"15min":0.501},"min":{"1min":0.38100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79700000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67300000000000004,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.98699999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.751},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.85099999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.873},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":456357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.31900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.41999999999999998,"15min":0.40100000000000002},"min":{"1min":0.33200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.80700000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63100000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78700000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.42299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.75800000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.443,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.439,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.625,"15min":0.624},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81699999999999995,"5min":0.874,"15min":0.874},"last":0.71899999999999997}]},{"osd":6,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.624,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:53:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77800000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.88800000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.492,"5min":5.492,"15min":5.492},"last":0.874}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":5,"up_from":37,"seq":158913789992,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1046885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68300000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":1.089,"15min":1.089},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86599999999999999,"5min":1.379,"15min":1.379},"last":0.628}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":1.407,"15min":1.407},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":1.476,"15min":1.476},"last":0.55800000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.54000000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.72599999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98599999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60999999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:53:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.754},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:53:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:54:25.667 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T09:54:25.667 INFO:tasks.cephadm:Setting up client nodes... 2023-10-08T09:54:25.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-08T09:54:26.256 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:26.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:26 smithi097 ceph-mon[99491]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:26.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:26 smithi178 ceph-mon[103156]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:26.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 09:54:26 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: 2023-10-08T09:54:26.165+0000 7f793dc9c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-08T09:54:26.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:26 smithi186 ceph-mon[104393]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:27 smithi097 ceph-mon[99491]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:27 smithi097 ceph-mon[99491]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-10-08T09:54:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:27 smithi178 ceph-mon[103156]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:27 smithi178 ceph-mon[103156]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-10-08T09:54:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:27 smithi186 ceph-mon[104393]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:54:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:27 smithi186 ceph-mon[104393]: pgmap v275: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.582+0000 7f6c4808b700 1 -- 172.21.15.97:0/1439920928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100330 msgr2=0x7f6c40107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.582+0000 7f6c4808b700 1 --2- 172.21.15.97:0/1439920928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100330 0x7f6c40107bd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800b0a0 tx=0x7f6c3800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.582+0000 7f6c4808b700 1 -- 172.21.15.97:0/1439920928 shutdown_connections 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.582+0000 7f6c4808b700 1 --2- 172.21.15.97:0/1439920928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100330 0x7f6c40107bd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 --2- 172.21.15.97:0/1439920928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100d10 0x7f6c40101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 --2- 172.21.15.97:0/1439920928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 0x7f6c400ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- 172.21.15.97:0/1439920928 >> 172.21.15.97:0/1439920928 conn(0x7f6c400fa420 msgr2=0x7f6c400fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- 172.21.15.97:0/1439920928 shutdown_connections 2023-10-08T09:54:27.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- 172.21.15.97:0/1439920928 wait complete. 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 Processor -- start 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- start start 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 0x7f6c400ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 0x7f6c4019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4019e2b0 con 0x7f6c40100d10 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4019e420 con 0x7f6c400ff970 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.583+0000 7f6c4808b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4019e5e0 con 0x7f6c40100330 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55644/0 (socket says 172.21.15.97:55644) 2023-10-08T09:54:27.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 -- 172.21.15.97:0/2583807892 learned_addr learned my addr 172.21.15.97:0/2583807892 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c45626700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 0x7f6c4019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c45e27700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 0x7f6c400ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 -- 172.21.15.97:0/2583807892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 msgr2=0x7f6c4019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 0x7f6c4019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 -- 172.21.15.97:0/2583807892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 msgr2=0x7f6c400ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 0x7f6c400ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 -- 172.21.15.97:0/2583807892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38009d00 con 0x7f6c40100d10 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c45626700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 0x7f6c4019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c46628700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004d00 tx=0x7f6c38004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800e040 con 0x7f6c40100d10 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4019e860 con 0x7f6c40100d10 2023-10-08T09:54:27.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c3801db50 con 0x7f6c40100d10 2023-10-08T09:54:27.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3801bbe0 con 0x7f6c40100d10 2023-10-08T09:54:27.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.584+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4019ecd0 con 0x7f6c40100d10 2023-10-08T09:54:27.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.585+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c40106bc0 con 0x7f6c40100d10 2023-10-08T09:54:27.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.587+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6c3801bd40 con 0x7f6c40100d10 2023-10-08T09:54:27.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.587+0000 7f6c36ffd700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 0x7f6c2c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:27.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.587+0000 7f6c45e27700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 0x7f6c2c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:27.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.587+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6c380cf0d0 con 0x7f6c40100d10 2023-10-08T09:54:27.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.588+0000 7f6c45e27700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 0x7f6c2c0a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:27.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.591+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6c380985d0 con 0x7f6c40100d10 2023-10-08T09:54:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.860+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6c4004e4c0 con 0x7f6c40100d10 2023-10-08T09:54:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.862+0000 7f6c36ffd700 1 -- 172.21.15.97:0/2583807892 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6c3809bbf0 con 0x7f6c40100d10 2023-10-08T09:54:27.863 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-10-08T09:54:27.863 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTfCJl+vdbMxAAWIKDSD3CH1WH+nIZhmtypw== 2023-10-08T09:54:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.864+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 msgr2=0x7f6c2c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.864+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 0x7f6c2c0a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c300058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.864+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 msgr2=0x7f6c4019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.864+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004d00 tx=0x7f6c38004de0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 shutdown_connections 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6c2c0a1480 0x7f6c2c0a3940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c40100d10 0x7f6c4019db10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c400ff970 0x7f6c400ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 --2- 172.21.15.97:0/2583807892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c40100330 0x7f6c4019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 >> 172.21.15.97:0/2583807892 conn(0x7f6c400fa420 msgr2=0x7f6c40103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 shutdown_connections 2023-10-08T09:54:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:27.865+0000 7f6c4808b700 1 -- 172.21.15.97:0/2583807892 wait complete. 2023-10-08T09:54:28.307 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:54:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-08T09:54:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T09:54:28.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-08T09:54:28.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:28 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:54:28.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:28 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:54:28.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:28 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:54:28.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:28 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:54:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:28 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:54:28.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:28 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2583807892' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:54:28.943 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:29.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:29 smithi097 ceph-mon[99491]: pgmap v276: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-10-08T09:54:29.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:29 smithi178 ceph-mon[103156]: pgmap v276: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-10-08T09:54:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:29 smithi186 ceph-mon[104393]: pgmap v276: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 -- 172.21.15.97:0/486289802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 msgr2=0x7fc26810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 --2- 172.21.15.97:0/486289802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26810d670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b0a0 tx=0x7fc26000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 -- 172.21.15.97:0/486289802 shutdown_connections 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 --2- 172.21.15.97:0/486289802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26810d670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 --2- 172.21.15.97:0/486289802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2681067a0 0x7fc268106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 --2- 172.21.15.97:0/486289802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc268068190 0x7fc2680685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 -- 172.21.15.97:0/486289802 >> 172.21.15.97:0/486289802 conn(0x7fc268075790 msgr2=0x7fc268075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 -- 172.21.15.97:0/486289802 shutdown_connections 2023-10-08T09:54:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.241+0000 7fc26e242700 1 -- 172.21.15.97:0/486289802 wait complete. 2023-10-08T09:54:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 Processor -- start 2023-10-08T09:54:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 -- start start 2023-10-08T09:54:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 0x7fc26819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2681067a0 0x7fc268198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268199500 con 0x7fc268068b30 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268199670 con 0x7fc2681067a0 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc26e242700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268199830 con 0x7fc268068190 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.242+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55654/0 (socket says 172.21.15.97:55654) 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 -- 172.21.15.97:0/2167150891 learned_addr learned my addr 172.21.15.97:0/2167150891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc267fff700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2681067a0 0x7fc268198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc2677fe700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 0x7fc26819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 -- 172.21.15.97:0/2167150891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 msgr2=0x7fc26819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 0x7fc26819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 -- 172.21.15.97:0/2167150891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2681067a0 msgr2=0x7fc268198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2681067a0 0x7fc268198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 -- 172.21.15.97:0/2167150891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009d00 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc2677fe700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 0x7fc26819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc266ffd700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc25800d950 tx=0x7fc25800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009940 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc268199b10 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc258010460 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f620 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.243+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2681a5c00 con 0x7fc268068b30 2023-10-08T09:54:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.244+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc268101ee0 con 0x7fc268068b30 2023-10-08T09:54:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.245+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc2580108d0 con 0x7fc268068b30 2023-10-08T09:54:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.246+0000 7fc264ff9700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 0x7fc2500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.246+0000 7fc2677fe700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 0x7fc2500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.246+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc2580cccc0 con 0x7fc268068b30 2023-10-08T09:54:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.246+0000 7fc2677fe700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 0x7fc2500a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009b50 tx=0x7fc25c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:30.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.250+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc258096130 con 0x7fc268068b30 2023-10-08T09:54:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.441+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7fc268100100 con 0x7fc268068b30 2023-10-08T09:54:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.442+0000 7fc264ff9700 1 -- 172.21.15.97:0/2167150891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7fc258016090 con 0x7fc268068b30 2023-10-08T09:54:30.443 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 msgr2=0x7fc2500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 0x7fc2500a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009b50 tx=0x7fc25c009550 comp rx=0 tx=0).stop 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 msgr2=0x7fc26819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc25800d950 tx=0x7fc25800dc60 comp rx=0 tx=0).stop 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 shutdown_connections 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc2500a1500 0x7fc2500a39c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2681067a0 0x7fc268198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc268068190 0x7fc26819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 --2- 172.21.15.97:0/2167150891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc268068b30 0x7fc26819f0c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 >> 172.21.15.97:0/2167150891 conn(0x7fc268075790 msgr2=0x7fc2680fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 shutdown_connections 2023-10-08T09:54:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:30.444+0000 7fc26e242700 1 -- 172.21.15.97:0/2167150891 wait complete. 2023-10-08T09:54:30.895 INFO:teuthology.orchestra.run.smithi097.stdout:[{"version":43,"timestamp":"2023-10-08T09:53:51.153305+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-08T09:54:30.895 INFO:tasks.ceph_manager:config epoch is 43 2023-10-08T09:54:30.896 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T09:54:30.896 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-08T09:54:30.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-08T09:54:31.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:31 smithi097 ceph-mon[99491]: pgmap v277: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-10-08T09:54:31.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:31 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2167150891' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:54:31.453 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:31 smithi178 ceph-mon[103156]: pgmap v277: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-10-08T09:54:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:31 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2167150891' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:54:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:31 smithi186 ceph-mon[104393]: pgmap v277: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-10-08T09:54:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:31 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2167150891' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:54:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- 172.21.15.97:0/1045977416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8104720 msgr2=0x7fd9b8104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/1045977416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8104720 0x7fd9b8104b00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009990 tx=0x7fd9b4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- 172.21.15.97:0/1045977416 shutdown_connections 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/1045977416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8100c60 0x7fd9b810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/1045977416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8104720 0x7fd9b8104b00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/1045977416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 0x7fd9b8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- 172.21.15.97:0/1045977416 >> 172.21.15.97:0/1045977416 conn(0x7fd9b8074a10 msgr2=0x7fd9b8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- 172.21.15.97:0/1045977416 shutdown_connections 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- 172.21.15.97:0/1045977416 wait complete. 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 Processor -- start 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.780+0000 7fd9c04fb700 1 -- start start 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 0x7fd9b8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:32.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 0x7fd9b81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81987d0 con 0x7fd9b8100c60 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81a3000 con 0x7fd9b81002a0 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81a3140 con 0x7fd9b8104720 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55668/0 (socket says 172.21.15.97:55668) 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 -- 172.21.15.97:0/2391759401 learned_addr learned my addr 172.21.15.97:0/2391759401 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9be297700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 0x7fd9b8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bea98700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 0x7fd9b81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 -- 172.21.15.97:0/2391759401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 msgr2=0x7fd9b81a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 0x7fd9b81a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 -- 172.21.15.97:0/2391759401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 msgr2=0x7fd9b8197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:32.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 0x7fd9b8197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 -- 172.21.15.97:0/2391759401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40095f0 con 0x7fd9b8100c60 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bea98700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 0x7fd9b81a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9bda96700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800d980 tx=0x7fd9a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8009b40 con 0x7fd9b8100c60 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b81a33a0 con 0x7fd9b8100c60 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9a800b6e0 con 0x7fd9b8100c60 2023-10-08T09:54:32.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800f620 con 0x7fd9b8100c60 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.781+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b81a38a0 con 0x7fd9b8100c60 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.782+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b8107c80 con 0x7fd9b8100c60 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.783+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd9a8010460 con 0x7fd9b8100c60 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.784+0000 7fd9af7fe700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 0x7fd9a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.784+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9a80cd180 con 0x7fd9b8100c60 2023-10-08T09:54:32.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.784+0000 7fd9be297700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 0x7fd9a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:32.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.784+0000 7fd9be297700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 0x7fd9a40a3940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400b5c0 tx=0x7fd9b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:32.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:32.788+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9a80965f0 con 0x7fd9b8100c60 2023-10-08T09:54:33.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.034+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd9b8067d10 con 0x7fd9b8100c60 2023-10-08T09:54:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.038+0000 7fd9af7fe700 1 -- 172.21.15.97:0/2391759401 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+260320 (secure 0 0 0) 0x7fd9a8099c10 con 0x7fd9b8100c60 2023-10-08T09:54:33.039 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 msgr2=0x7fd9a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 0x7fd9a40a3940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400b5c0 tx=0x7fd9b4005960 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 msgr2=0x7fd9b8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800d980 tx=0x7fd9a800dc90 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 shutdown_connections 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd9a40a1480 0x7fd9a40a3940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b8104720 0x7fd9b81a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b81002a0 0x7fd9b8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 --2- 172.21.15.97:0/2391759401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b8100c60 0x7fd9b8197f80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 >> 172.21.15.97:0/2391759401 conn(0x7fd9b8074a10 msgr2=0x7fd9b80fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 shutdown_connections 2023-10-08T09:54:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:33.041+0000 7fd9c04fb700 1 -- 172.21.15.97:0/2391759401 wait complete. 2023-10-08T09:54:33.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:33 smithi097 ceph-mon[99491]: pgmap v278: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:33.439 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:33 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2391759401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:54:33.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:33 smithi178 ceph-mon[103156]: pgmap v278: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:33.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:33 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2391759401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:54:33.500 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3991091613},{"type":"v1","addr":"172.21.15.97:6801","nonce":3991091613}]},"active_addr":"172.21.15.97:6801/3991091613","active_change":"2023-10-08T09:47:45.797769+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2644603845}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2446646431}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4092341573}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3419108156}]}]} 2023-10-08T09:54:33.506 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-08T09:54:33.506 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-08T09:54:33.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:54:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:33 smithi186 ceph-mon[104393]: pgmap v278: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:33 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2391759401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:54:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 -- 172.21.15.97:0/1445642644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9841056b0 msgr2=0x7fa984105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 --2- 172.21.15.97:0/1445642644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9841056b0 0x7fa984105a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa978009990 tx=0x7fa978009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 -- 172.21.15.97:0/1445642644 shutdown_connections 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 --2- 172.21.15.97:0/1445642644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 0x7fa98410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 --2- 172.21.15.97:0/1445642644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9841056b0 0x7fa984105a90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 --2- 172.21.15.97:0/1445642644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9840ff750 0x7fa9840ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 -- 172.21.15.97:0/1445642644 >> 172.21.15.97:0/1445642644 conn(0x7fa984075790 msgr2=0x7fa984075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 -- 172.21.15.97:0/1445642644 shutdown_connections 2023-10-08T09:54:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 -- 172.21.15.97:0/1445642644 wait complete. 2023-10-08T09:54:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.377+0000 7fa989246700 1 Processor -- start 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 -- start start 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 0x7fa984197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 0x7fa9841a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa984198810 con 0x7fa9840ff750 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841a2fd0 con 0x7fa984100110 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa989246700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841a3110 con 0x7fa9841056b0 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55680/0 (socket says 172.21.15.97:55680) 2023-10-08T09:54:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 -- 172.21.15.97:0/4242634258 learned_addr learned my addr 172.21.15.97:0/4242634258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa98259c700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 0x7fa984197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa98359e700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 0x7fa9841a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 -- 172.21.15.97:0/4242634258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 msgr2=0x7fa9841a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 0x7fa9841a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 -- 172.21.15.97:0/4242634258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 msgr2=0x7fa984197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 0x7fa984197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.378+0000 7fa982d9d700 1 -- 172.21.15.97:0/4242634258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780095f0 con 0x7fa9840ff750 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa98359e700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 0x7fa9841a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa982d9d700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa978009fd0 tx=0x7fa97800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97801d070 con 0x7fa9840ff750 2023-10-08T09:54:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9841a3330 con 0x7fa9840ff750 2023-10-08T09:54:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa978005510 con 0x7fa9840ff750 2023-10-08T09:54:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978004a90 con 0x7fa9840ff750 2023-10-08T09:54:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.379+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9841a37d0 con 0x7fa9840ff750 2023-10-08T09:54:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.380+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa984107c40 con 0x7fa9840ff750 2023-10-08T09:54:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.381+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa978004c70 con 0x7fa9840ff750 2023-10-08T09:54:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.381+0000 7fa96bfff700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 0x7fa96c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.382+0000 7fa98259c700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 0x7fa96c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.382+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9780cdc10 con 0x7fa9840ff750 2023-10-08T09:54:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.382+0000 7fa98259c700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 0x7fa96c0a3940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9840689a0 tx=0x7fa97400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.386+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa978097110 con 0x7fa9840ff750 2023-10-08T09:54:35.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:35 smithi097 ceph-mon[99491]: pgmap v279: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:35.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:35 smithi178 ceph-mon[103156]: pgmap v279: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:35 smithi186 ceph-mon[104393]: pgmap v279: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2023-10-08T09:54:35.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.580+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa984068750 con 0x7fa9840ff750 2023-10-08T09:54:35.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.581+0000 7fa96bfff700 1 -- 172.21.15.97:0/4242634258 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7fa97809a730 con 0x7fa9840ff750 2023-10-08T09:54:35.583 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:35.583 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:35.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 msgr2=0x7fa96c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 0x7fa96c0a3940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9840689a0 tx=0x7fa97400a300 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 msgr2=0x7fa984197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa978009fd0 tx=0x7fa97800f670 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 shutdown_connections 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa96c0a1480 0x7fa96c0a3940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9841056b0 0x7fa9841a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9840ff750 0x7fa984197a10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 --2- 172.21.15.97:0/4242634258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984100110 0x7fa984197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 >> 172.21.15.97:0/4242634258 conn(0x7fa984075790 msgr2=0x7fa9840fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:35.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 shutdown_connections 2023-10-08T09:54:35.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:35.584+0000 7fa989246700 1 -- 172.21.15.97:0/4242634258 wait complete. 2023-10-08T09:54:36.024 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-08T09:54:36.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:54:36.211 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:36 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4242634258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:36.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:36 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4242634258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:36.581 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:36 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4242634258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:37.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:37 smithi097 ceph-mon[99491]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2023-10-08T09:54:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:37 smithi186 ceph-mon[104393]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2023-10-08T09:54:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:37 smithi178 ceph-mon[103156]: pgmap v280: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 -- 172.21.15.97:0/4241392158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 msgr2=0x7fb43c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 --2- 172.21.15.97:0/4241392158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c105aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009990 tx=0x7fb42c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 -- 172.21.15.97:0/4241392158 shutdown_connections 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 --2- 172.21.15.97:0/4241392158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c100810 0x7fb43c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 --2- 172.21.15.97:0/4241392158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c105aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 --2- 172.21.15.97:0/4241392158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c0ffe50 0x7fb43c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 -- 172.21.15.97:0/4241392158 >> 172.21.15.97:0/4241392158 conn(0x7fb43c075cf0 msgr2=0x7fb43c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 -- 172.21.15.97:0/4241392158 shutdown_connections 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.914+0000 7fb443aec700 1 -- 172.21.15.97:0/4241392158 wait complete. 2023-10-08T09:54:37.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 Processor -- start 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 -- start start 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 0x7fb43c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c100810 0x7fb43c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1987c0 con 0x7fb43c1056c0 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1a2ff0 con 0x7fb43c100810 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb443aec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1a3130 con 0x7fb43c0ffe50 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55696/0 (socket says 172.21.15.97:55696) 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 -- 172.21.15.97:0/120763105 learned_addr learned my addr 172.21.15.97:0/120763105 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:37.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb441087700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c100810 0x7fb43c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb441888700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 0x7fb43c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 -- 172.21.15.97:0/120763105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 msgr2=0x7fb43c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 0x7fb43c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 -- 172.21.15.97:0/120763105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c100810 msgr2=0x7fb43c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c100810 0x7fb43c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb442089700 1 -- 172.21.15.97:0/120763105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c0095f0 con 0x7fb43c1056c0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.915+0000 7fb441888700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 0x7fb43c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb442089700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb434007f10 tx=0x7fb43400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400ddd0 con 0x7fb43c1056c0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43400f040 con 0x7fb43c1056c0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c1a3390 con 0x7fb43c1056c0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434015400 con 0x7fb43c1056c0 2023-10-08T09:54:37.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.916+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c1a3840 con 0x7fb43c1056c0 2023-10-08T09:54:37.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.917+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c107cc0 con 0x7fb43c1056c0 2023-10-08T09:54:37.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.918+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb434004000 con 0x7fb43c1056c0 2023-10-08T09:54:37.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.918+0000 7fb432ffd700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 0x7fb4280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:37.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.918+0000 7fb441888700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 0x7fb4280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:37.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.919+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb4340ccfe0 con 0x7fb43c1056c0 2023-10-08T09:54:37.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.919+0000 7fb441888700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 0x7fb4280a3940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:37.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:37.922+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb434096450 con 0x7fb43c1056c0 2023-10-08T09:54:38.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.111+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb43c100e60 con 0x7fb43c1056c0 2023-10-08T09:54:38.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.112+0000 7fb432ffd700 1 -- 172.21.15.97:0/120763105 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23973 (secure 0 0 0) 0x7fb434023090 con 0x7fb43c1056c0 2023-10-08T09:54:38.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:54:38.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:01.622943+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:54:38.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 msgr2=0x7fb4280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:38.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 0x7fb4280a3940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c01a040 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 msgr2=0x7fb43c1a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb434007f10 tx=0x7fb43400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 shutdown_connections 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4280a1480 0x7fb4280a3940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43c1056c0 0x7fb43c1a2ab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb43c0ffe50 0x7fb43c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 --2- 172.21.15.97:0/120763105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43c100810 0x7fb43c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 >> 172.21.15.97:0/120763105 conn(0x7fb43c075cf0 msgr2=0x7fb43c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 shutdown_connections 2023-10-08T09:54:38.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:38.115+0000 7fb443aec700 1 -- 172.21.15.97:0/120763105 wait complete. 2023-10-08T09:54:38.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/120763105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:38.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/120763105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:38.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-08T09:54:38.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-08T09:54:38.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-08T09:54:38.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-08T09:54:38.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-08T09:54:38.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-08T09:54:38.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-08T09:54:38.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-08T09:54:38.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-08T09:54:38.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-08T09:54:38.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-08T09:54:38.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-08T09:54:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/120763105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:54:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:39 smithi186 ceph-mon[104393]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-08T09:54:39.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:39 smithi097 ceph-mon[99491]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-08T09:54:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:39 smithi178 ceph-mon[103156]: pgmap v281: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-08T09:54:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:39.962 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:40.148 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:40.263 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:40.362 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:40.363 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:41.366 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:41 smithi097 ceph-mon[99491]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-08T09:54:41.377 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:41 smithi186 ceph-mon[104393]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-08T09:54:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:41 smithi178 ceph-mon[103156]: pgmap v282: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-08T09:54:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:42.971 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:43.085 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:43.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 -- 172.21.15.97:0/1857224981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf94101200 msgr2=0x7faf9410d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 --2- 172.21.15.97:0/1857224981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf94101200 0x7faf9410d630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b0a0 tx=0x7faf8c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 -- 172.21.15.97:0/1857224981 shutdown_connections 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 --2- 172.21.15.97:0/1857224981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf94101200 0x7faf9410d630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 --2- 172.21.15.97:0/1857224981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf941056a0 0x7faf94105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 --2- 172.21.15.97:0/1857224981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf94100840 0x7faf94100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 -- 172.21.15.97:0/1857224981 >> 172.21.15.97:0/1857224981 conn(0x7faf94077950 msgr2=0x7faf94077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 -- 172.21.15.97:0/1857224981 shutdown_connections 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.001+0000 7faf99053700 1 -- 172.21.15.97:0/1857224981 wait complete. 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 Processor -- start 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 -- start start 2023-10-08T09:54:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf94100840 0x7faf9419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf94101200 0x7faf9419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 0x7faf94196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf94197270 con 0x7faf941056a0 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf941973e0 con 0x7faf94101200 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf99053700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf941975a0 con 0x7faf94100840 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf9359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 0x7faf94196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf9359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 0x7faf94196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55710/0 (socket says 172.21.15.97:55710) 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf9359e700 1 -- 172.21.15.97:0/3390221611 learned_addr learned my addr 172.21.15.97:0/3390221611 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf9259c700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf94101200 0x7faf9419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.002+0000 7faf92d9d700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf94100840 0x7faf9419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 -- 172.21.15.97:0/3390221611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf94100840 msgr2=0x7faf9419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf94100840 0x7faf9419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 -- 172.21.15.97:0/3390221611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf94101200 msgr2=0x7faf9419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf94101200 0x7faf9419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009d00 con 0x7faf941056a0 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf9359e700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 0x7faf94196ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b070 tx=0x7faf8c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00e040 con 0x7faf941056a0 2023-10-08T09:54:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf94197820 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf8c009440 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01b450 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.003+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf941a3990 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.004+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf941109c0 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.005+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faf8c01d480 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf7bfff700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf7c0a1480 0x7faf7c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faf8c0cee70 con 0x7faf941056a0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf92d9d700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf7c0a1480 0x7faf7c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf7bfff700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] conn(0x7faf7c0ac000 0x7faf7c0ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf7c0aeb10 con 0x7faf7c0ac000 2023-10-08T09:54:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf8c0cf280 con 0x7faf941056a0 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf9259c700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] conn(0x7faf7c0ac000 0x7faf7c0ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.006+0000 7faf92d9d700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf7c0a1480 0x7faf7c0a3940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf88007b60 tx=0x7faf880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.007+0000 7faf9259c700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] conn(0x7faf7c0ac000 0x7faf7c0ae440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.007+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== osd.6 v2:172.21.15.97:6818/3267640836 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7faf7c0aeb10 con 0x7faf7c0ac000 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.023+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 --> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf94060ef0 con 0x7faf7c0ac000 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf7bfff700 1 -- 172.21.15.97:0/3390221611 <== osd.6 v2:172.21.15.97:6818/3267640836 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faf94060ef0 con 0x7faf7c0ac000 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] conn(0x7faf7c0ac000 msgr2=0x7faf7c0ae440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] conn(0x7faf7c0ac000 0x7faf7c0ae440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf7c0a1480 msgr2=0x7faf7c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf7c0a1480 0x7faf7c0a3940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf88007b60 tx=0x7faf880073d0 comp rx=0 tx=0).stop 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 msgr2=0x7faf94196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf99053700 1 --2- 172.21.15.97:0/3390221611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf941056a0 0x7faf94196ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b070 tx=0x7faf8c007870 comp rx=0 tx=0).stop 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.024+0000 7faf9359e700 1 -- 172.21.15.97:0/3390221611 reap_dead start 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.025+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 shutdown_connections 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.025+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 >> 172.21.15.97:0/3390221611 conn(0x7faf94077950 msgr2=0x7faf940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.025+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 shutdown_connections 2023-10-08T09:54:43.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:43.025+0000 7faf99053700 1 -- 172.21.15.97:0/3390221611 wait complete. 2023-10-08T09:54:43.484 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:43 smithi097 ceph-mon[99491]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:54:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:43 smithi186 ceph-mon[104393]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:54:43.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:43 smithi178 ceph-mon[103156]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:54:43.991 INFO:teuthology.orchestra.run.smithi097.stdout:188978561065 2023-10-08T09:54:43.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-08T09:54:45.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:45 smithi097 ceph-mon[99491]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-08T09:54:45.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:45 smithi186 ceph-mon[104393]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-08T09:54:45.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:45 smithi178 ceph-mon[103156]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-08T09:54:46.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 -- 172.21.15.97:0/2391579047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc101200 msgr2=0x7ff0fc10d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 --2- 172.21.15.97:0/2391579047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc101200 0x7ff0fc10d6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b0a0 tx=0x7ff0f400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 -- 172.21.15.97:0/2391579047 shutdown_connections 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 --2- 172.21.15.97:0/2391579047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc101200 0x7ff0fc10d6a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 --2- 172.21.15.97:0/2391579047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 --2- 172.21.15.97:0/2391579047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0fc100840 0x7ff0fc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 -- 172.21.15.97:0/2391579047 >> 172.21.15.97:0/2391579047 conn(0x7ff0fc077950 msgr2=0x7ff0fc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 -- 172.21.15.97:0/2391579047 shutdown_connections 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.076+0000 7ff10132e700 1 -- 172.21.15.97:0/2391579047 wait complete. 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 Processor -- start 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 -- start start 2023-10-08T09:54:46.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0fc100840 0x7ff0fc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc101200 0x7ff0fc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc197310 con 0x7ff0fc1057a0 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc197480 con 0x7ff0fc101200 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.077+0000 7ff10132e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc197640 con 0x7ff0fc100840 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55742/0 (socket says 172.21.15.97:55742) 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 -- 172.21.15.97:0/2306586984 learned_addr learned my addr 172.21.15.97:0/2306586984 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0faffd700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0fc100840 0x7ff0fc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fa7fc700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc101200 0x7ff0fc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 -- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0fc100840 msgr2=0x7ff0fc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0fc100840 0x7ff0fc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 -- 172.21.15.97:0/2306586984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc101200 msgr2=0x7ff0fc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc101200 0x7ff0fc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009d00 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0fb7fe700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc196b60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b070 tx=0x7ff0f4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400e040 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc1978c0 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f4009440 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f401b450 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.078+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1a39e0 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.079+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0fc110a00 con 0x7ff0fc1057a0 2023-10-08T09:54:46.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.080+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff0f401d480 con 0x7ff0fc1057a0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0e3fff700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e40a1480 0x7ff0e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0faffd700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e40a1480 0x7ff0e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0f40cee70 con 0x7ff0fc1057a0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0e3fff700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] conn(0x7ff0e40ac000 0x7ff0e40ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0e40aeb10 con 0x7ff0e40ac000 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f40cf280 con 0x7ff0fc1057a0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0faffd700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e40a1480 0x7ff0e40a3940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00abd0 tx=0x7ff0ec00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.081+0000 7ff0fa7fc700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] conn(0x7ff0e40ac000 0x7ff0e40ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.082+0000 7ff0fa7fc700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] conn(0x7ff0e40ac000 0x7ff0e40ae440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.083+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== osd.5 v2:172.21.15.186:6808/2325993669 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff0e40aeb10 con 0x7ff0e40ac000 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.099+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 --> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0fc060e70 con 0x7ff0e40ac000 2023-10-08T09:54:46.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff0e3fff700 1 -- 172.21.15.97:0/2306586984 <== osd.5 v2:172.21.15.186:6808/2325993669 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff0fc060e70 con 0x7ff0e40ac000 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] conn(0x7ff0e40ac000 msgr2=0x7ff0e40ae440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] conn(0x7ff0e40ac000 0x7ff0e40ae440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e40a1480 msgr2=0x7ff0e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0e40a1480 0x7ff0e40a3940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00abd0 tx=0x7ff0ec00a380 comp rx=0 tx=0).stop 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 msgr2=0x7ff0fc196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 --2- 172.21.15.97:0/2306586984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc1057a0 0x7ff0fc196b60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b070 tx=0x7ff0f4007870 comp rx=0 tx=0).stop 2023-10-08T09:54:46.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff0fb7fe700 1 -- 172.21.15.97:0/2306586984 reap_dead start 2023-10-08T09:54:46.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 shutdown_connections 2023-10-08T09:54:46.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 >> 172.21.15.97:0/2306586984 conn(0x7ff0fc077950 msgr2=0x7ff0fc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:46.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 shutdown_connections 2023-10-08T09:54:46.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:46.100+0000 7ff10132e700 1 -- 172.21.15.97:0/2306586984 wait complete. 2023-10-08T09:54:46.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 -- 172.21.15.97:0/3880877911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 msgr2=0x7f54b010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 --2- 172.21.15.97:0/3880877911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b010c580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b0a0 tx=0x7f54a800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 -- 172.21.15.97:0/3880877911 shutdown_connections 2023-10-08T09:54:46.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 --2- 172.21.15.97:0/3880877911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b010c580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 --2- 172.21.15.97:0/3880877911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b01056b0 0x7f54b0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 --2- 172.21.15.97:0/3880877911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b00ff750 0x7f54b00ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 -- 172.21.15.97:0/3880877911 >> 172.21.15.97:0/3880877911 conn(0x7f54b0075790 msgr2=0x7f54b0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 -- 172.21.15.97:0/3880877911 shutdown_connections 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.148+0000 7f54b4f37700 1 -- 172.21.15.97:0/3880877911 wait complete. 2023-10-08T09:54:46.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 Processor -- start 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 -- start start 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b00ff750 0x7f54b0196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b01056b0 0x7f54b019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019bc80 con 0x7f54b0100110 2023-10-08T09:54:46.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019bdf0 con 0x7f54b01056b0 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54b4f37700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b019bfb0 con 0x7f54b00ff750 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b0197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55724/0 (socket says 172.21.15.97:55724) 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 -- 172.21.15.97:0/539141518 learned_addr learned my addr 172.21.15.97:0/539141518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54aed9d700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b01056b0 0x7f54b019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54ae59c700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b00ff750 0x7f54b0196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 -- 172.21.15.97:0/539141518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b00ff750 msgr2=0x7f54b0196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b00ff750 0x7f54b0196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 -- 172.21.15.97:0/539141518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b01056b0 msgr2=0x7f54b019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54b01056b0 0x7f54b019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54add9b700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8009d00 con 0x7f54b0100110 2023-10-08T09:54:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.149+0000 7f54ae59c700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f54b00ff750 0x7f54b0196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:46.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f54add9b700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b0197310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54a400d980 tx=0x7f54a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004a30 con 0x7f54b0100110 2023-10-08T09:54:46.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54a40054e0 con 0x7f54b0100110 2023-10-08T09:54:46.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b019c290 con 0x7f54b0100110 2023-10-08T09:54:46.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4009c90 con 0x7f54b0100110 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.150+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b01a3af0 con 0x7f54b0100110 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.151+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54b010f960 con 0x7f54b0100110 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.152+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f54a4004c20 con 0x7f54b0100110 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.152+0000 7f549f7fe700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f54980a1480 0x7f54980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.152+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54a40cd360 con 0x7f54b0100110 2023-10-08T09:54:46.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.152+0000 7f54ae59c700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f54980a1480 0x7f54980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f549f7fe700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] conn(0x7f54980ac000 0x7f54980ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54980aeb10 con 0x7f54980ac000 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a40cd770 con 0x7f54b0100110 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f54aed9d700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] conn(0x7f54980ac000 0x7f54980ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f54ae59c700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f54980a1480 0x7f54980a3940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007900 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.153+0000 7f54aed9d700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] conn(0x7f54980ac000 0x7f54980ae440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.154+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== osd.3 v2:172.21.15.97:6810/105487765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f54980aeb10 con 0x7f54980ac000 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.170+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 --> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54b0060ef0 con 0x7f54980ac000 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f549f7fe700 1 -- 172.21.15.97:0/539141518 <== osd.3 v2:172.21.15.97:6810/105487765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f54b0060ef0 con 0x7f54980ac000 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] conn(0x7f54980ac000 msgr2=0x7f54980ae440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] conn(0x7f54980ac000 0x7f54980ae440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f54980a1480 msgr2=0x7f54980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f54980a1480 0x7f54980a3940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54a0007900 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 msgr2=0x7f54b0197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 --2- 172.21.15.97:0/539141518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b0100110 0x7f54b0197310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54a400d980 tx=0x7f54a400dc90 comp rx=0 tx=0).stop 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54aed9d700 1 -- 172.21.15.97:0/539141518 reap_dead start 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 shutdown_connections 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 >> 172.21.15.97:0/539141518 conn(0x7f54b0075790 msgr2=0x7f54b0068530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 shutdown_connections 2023-10-08T09:54:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:45.171+0000 7f54b4f37700 1 -- 172.21.15.97:0/539141518 wait complete. 2023-10-08T09:54:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2023-10-08T09:54:46.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-10-08T09:54:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T09:54:46.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2023-10-08T09:54:46.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-10-08T09:54:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T09:54:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2023-10-08T09:54:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2023-10-08T09:54:46.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-10-08T09:54:47.136 INFO:teuthology.orchestra.run.smithi097.stdout:107374182453 2023-10-08T09:54:47.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-08T09:54:47.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 -- 172.21.15.97:0/2244743285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 msgr2=0x7f3a6c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 --2- 172.21.15.97:0/2244743285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 0x7f3a6c100cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00b0a0 tx=0x7f3a5c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 -- 172.21.15.97:0/2244743285 shutdown_connections 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 --2- 172.21.15.97:0/2244743285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a6c101200 0x7f3a6c10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 --2- 172.21.15.97:0/2244743285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c1056a0 0x7f3a6c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 --2- 172.21.15.97:0/2244743285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 0x7f3a6c100cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 -- 172.21.15.97:0/2244743285 >> 172.21.15.97:0/2244743285 conn(0x7f3a6c077950 msgr2=0x7f3a6c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:47.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 -- 172.21.15.97:0/2244743285 shutdown_connections 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.074+0000 7f3a739d4700 1 -- 172.21.15.97:0/2244743285 wait complete. 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.075+0000 7f3a739d4700 1 Processor -- start 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.075+0000 7f3a739d4700 1 -- start start 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.075+0000 7f3a739d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 0x7f3a6c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.075+0000 7f3a739d4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 0x7f3a6c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a6c1056a0 0x7f3a6c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c19bcf0 con 0x7f3a6c101200 2023-10-08T09:54:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c19be60 con 0x7f3a6c1056a0 2023-10-08T09:54:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c19c020 con 0x7f3a6c100840 2023-10-08T09:54:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 0x7f3a6c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 0x7f3a6c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55756/0 (socket says 172.21.15.97:55756) 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 -- 172.21.15.97:0/3379989732 learned_addr learned my addr 172.21.15.97:0/3379989732 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a71770700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 0x7f3a6c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a71f71700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a6c1056a0 0x7f3a6c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 -- 172.21.15.97:0/3379989732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 msgr2=0x7f3a6c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3a6c100840 0x7f3a6c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 -- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a6c1056a0 msgr2=0x7f3a6c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a6c1056a0 0x7f3a6c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c009d00 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a70f6f700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 0x7f3a6c197330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00b070 tx=0x7f3a5c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00e040 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c19c2a0 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a5c004e60 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c005400 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.076+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c1a3b60 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.077+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a6c110a40 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.078+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3a5c01baa0 con 0x7f3a6c101200 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a6a7fc700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3a540a1480 0x7f3a540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a71770700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3a540a1480 0x7f3a540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a5c0ce940 con 0x7f3a6c101200 2023-10-08T09:54:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a6a7fc700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] conn(0x7f3a540ac4a0 0x7f3a540ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a540af010 con 0x7f3a540ac4a0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a5c0ced50 con 0x7f3a6c101200 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a71770700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3a540a1480 0x7f3a540a3940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009a10 tx=0x7f3a64009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.079+0000 7f3a71f71700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] conn(0x7f3a540ac4a0 0x7f3a540ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.080+0000 7f3a71f71700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] conn(0x7f3a540ac4a0 0x7f3a540ae940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.081+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== osd.10 v2:172.21.15.178:6824/3571291491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3a540af010 con 0x7f3a540ac4a0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.097+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 --> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a6c060e70 con 0x7f3a540ac4a0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.097+0000 7f3a6a7fc700 1 -- 172.21.15.97:0/3379989732 <== osd.10 v2:172.21.15.178:6824/3571291491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a6c060e70 con 0x7f3a540ac4a0 2023-10-08T09:54:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] conn(0x7f3a540ac4a0 msgr2=0x7f3a540ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] conn(0x7f3a540ac4a0 0x7f3a540ae940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3a540a1480 msgr2=0x7f3a540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3a540a1480 0x7f3a540a3940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009a10 tx=0x7f3a64009190 comp rx=0 tx=0).stop 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 msgr2=0x7f3a6c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 --2- 172.21.15.97:0/3379989732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a6c101200 0x7f3a6c197330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00b070 tx=0x7f3a5c007870 comp rx=0 tx=0).stop 2023-10-08T09:54:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a71f71700 1 -- 172.21.15.97:0/3379989732 reap_dead start 2023-10-08T09:54:47.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 shutdown_connections 2023-10-08T09:54:47.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 >> 172.21.15.97:0/3379989732 conn(0x7f3a6c077950 msgr2=0x7f3a6c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:47.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 shutdown_connections 2023-10-08T09:54:47.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.098+0000 7f3a739d4700 1 -- 172.21.15.97:0/3379989732 wait complete. 2023-10-08T09:54:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 -- 172.21.15.97:0/502751579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384069450 msgr2=0x7f938410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 --2- 172.21.15.97:0/502751579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384069450 0x7f938410d650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f93740099c0 tx=0x7f9374009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 -- 172.21.15.97:0/502751579 shutdown_connections 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 --2- 172.21.15.97:0/502751579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384069450 0x7f938410d650 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 --2- 172.21.15.97:0/502751579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9384105670 0x7f9384105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 --2- 172.21.15.97:0/502751579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9384068a90 0x7f9384068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 -- 172.21.15.97:0/502751579 >> 172.21.15.97:0/502751579 conn(0x7f93840766c0 msgr2=0x7f9384076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 -- 172.21.15.97:0/502751579 shutdown_connections 2023-10-08T09:54:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.383+0000 7f9388fe0700 1 -- 172.21.15.97:0/502751579 wait complete. 2023-10-08T09:54:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 Processor -- start 2023-10-08T09:54:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 -- start start 2023-10-08T09:54:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 0x7f9384198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9384069450 0x7f93841993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9384105670 0x7f938419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938419dd30 con 0x7f9384068a90 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938419dea0 con 0x7f9384069450 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9388fe0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938419e060 con 0x7f9384105670 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 0x7f9384198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 0x7f9384198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55770/0 (socket says 172.21.15.97:55770) 2023-10-08T09:54:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 -- 172.21.15.97:0/597652179 learned_addr learned my addr 172.21.15.97:0/597652179 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9381d9b700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9384069450 0x7f93841993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f9382d9d700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9384105670 0x7f938419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 -- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9384105670 msgr2=0x7f938419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9384105670 0x7f938419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 -- 172.21.15.97:0/597652179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9384069450 msgr2=0x7f93841993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9384069450 0x7f93841993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.384+0000 7f938259c700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740095f0 con 0x7f9384068a90 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f938259c700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 0x7f9384198e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f937c00ba50 tx=0x7f937c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00cab0 con 0x7f9384068a90 2023-10-08T09:54:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938419e340 con 0x7f9384068a90 2023-10-08T09:54:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f937c00cc10 con 0x7f9384068a90 2023-10-08T09:54:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c0075d0 con 0x7f9384068a90 2023-10-08T09:54:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.385+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93841a5ba0 con 0x7f9384068a90 2023-10-08T09:54:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.386+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9384110a70 con 0x7f9384068a90 2023-10-08T09:54:47.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.387+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f937c014440 con 0x7f9384068a90 2023-10-08T09:54:47.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.387+0000 7f937b7fe700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f93700a1480 0x7f93700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.387+0000 7f9381d9b700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f93700a1480 0x7f93700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f937c006e40 con 0x7f9384068a90 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f937b7fe700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] conn(0x7f93700ac4a0 0x7f93700ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93700af010 con 0x7f93700ac4a0 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937c0cd570 con 0x7f9384068a90 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f9381d9b700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f93700a1480 0x7f93700a3940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f9382d9d700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] conn(0x7f93700ac4a0 0x7f93700ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.388+0000 7f9382d9d700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] conn(0x7f93700ac4a0 0x7f93700ae940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.389+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== osd.8 v2:172.21.15.186:6816/426737999 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f93700af010 con 0x7f93700ac4a0 2023-10-08T09:54:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.406+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 --> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9384101ee0 con 0x7f93700ac4a0 2023-10-08T09:54:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.406+0000 7f937b7fe700 1 -- 172.21.15.97:0/597652179 <== osd.8 v2:172.21.15.186:6816/426737999 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9384101ee0 con 0x7f93700ac4a0 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] conn(0x7f93700ac4a0 msgr2=0x7f93700ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] conn(0x7f93700ac4a0 0x7f93700ae940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f93700a1480 msgr2=0x7f93700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f93700a1480 0x7f93700a3940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c008040 comp rx=0 tx=0).stop 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 msgr2=0x7f9384198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 --2- 172.21.15.97:0/597652179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9384068a90 0x7f9384198e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f937c00ba50 tx=0x7f937c00be10 comp rx=0 tx=0).stop 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9382d9d700 1 -- 172.21.15.97:0/597652179 reap_dead start 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 shutdown_connections 2023-10-08T09:54:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 >> 172.21.15.97:0/597652179 conn(0x7f93840766c0 msgr2=0x7f938410bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 shutdown_connections 2023-10-08T09:54:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:47.407+0000 7f9388fe0700 1 -- 172.21.15.97:0/597652179 wait complete. 2023-10-08T09:54:47.429 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:54:47.429 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T09:54:47.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2023-10-08T09:54:47.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-10-08T09:54:47.431 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:47 smithi097 ceph-mon[99491]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-08T09:54:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-10-08T09:54:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T09:54:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2023-10-08T09:54:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-10-08T09:54:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:47 smithi186 ceph-mon[104393]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:54:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:54:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-08T09:54:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T09:54:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-10-08T09:54:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:47 smithi178 ceph-mon[103156]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:54:48.358 INFO:teuthology.orchestra.run.smithi097.stdout:292057776153 2023-10-08T09:54:48.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-08T09:54:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:48 smithi186 ceph-mon[104393]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:54:48.592 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:48 smithi097 ceph-mon[99491]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:54:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:48 smithi178 ceph-mon[103156]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:54:48.756 INFO:teuthology.orchestra.run.smithi097.stdout:158913789998 2023-10-08T09:54:48.756 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-08T09:54:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 -- 172.21.15.97:0/3231165518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c0ff750 msgr2=0x7fe58c0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 --2- 172.21.15.97:0/3231165518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c0ff750 0x7fe58c0ffbd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe580009990 tx=0x7fe580009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 -- 172.21.15.97:0/3231165518 shutdown_connections 2023-10-08T09:54:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 --2- 172.21.15.97:0/3231165518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c100110 0x7fe58c10c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 --2- 172.21.15.97:0/3231165518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe58c105690 0x7fe58c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 --2- 172.21.15.97:0/3231165518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c0ff750 0x7fe58c0ffbd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 -- 172.21.15.97:0/3231165518 >> 172.21.15.97:0/3231165518 conn(0x7fe58c0757d0 msgr2=0x7fe58c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 -- 172.21.15.97:0/3231165518 shutdown_connections 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.778+0000 7fe592428700 1 -- 172.21.15.97:0/3231165518 wait complete. 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.779+0000 7fe592428700 1 Processor -- start 2023-10-08T09:54:48.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.779+0000 7fe592428700 1 -- start start 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe58c0ff750 0x7fe58c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c100110 0x7fe58c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 0x7fe58c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c1970b0 con 0x7fe58c105690 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c197220 con 0x7fe58c100110 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe592428700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c1973e0 con 0x7fe58c0ff750 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 0x7fe58c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 0x7fe58c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55822/0 (socket says 172.21.15.97:55822) 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 -- 172.21.15.97:0/4256769783 learned_addr learned my addr 172.21.15.97:0/4256769783 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe58b7fe700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c100110 0x7fe58c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe58bfff700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe58c0ff750 0x7fe58c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 -- 172.21.15.97:0/4256769783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe58c0ff750 msgr2=0x7fe58c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe58c0ff750 0x7fe58c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 -- 172.21.15.97:0/4256769783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c100110 msgr2=0x7fe58c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe58c100110 0x7fe58c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800095f0 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5909c5700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 0x7fe58c196a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe584007ee0 tx=0x7fe58400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.780+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400ddd0 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.781+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58400f040 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.781+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c1976c0 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.781+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5840048e0 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.781+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c1a3900 con 0x7fe58c105690 2023-10-08T09:54:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.781+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe58c10f940 con 0x7fe58c105690 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.783+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe584003a00 con 0x7fe58c105690 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.783+0000 7fe5897fa700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5740a1500 0x7fe5740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.783+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5840cd420 con 0x7fe58c105690 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.783+0000 7fe58bfff700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5740a1500 0x7fe5740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe5897fa700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] conn(0x7fe5740ac3b0 0x7fe5740ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5740aef20 con 0x7fe5740ac3b0 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5840cd830 con 0x7fe58c105690 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe58b7fe700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] conn(0x7fe5740ac3b0 0x7fe5740ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe58bfff700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5740a1500 0x7fe5740a39c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005950 tx=0x7fe57c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.784+0000 7fe58b7fe700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] conn(0x7fe5740ac3b0 0x7fe5740ae850 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.785+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== osd.0 v2:172.21.15.97:6802/3104991007 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe5740aef20 con 0x7fe5740ac3b0 2023-10-08T09:54:48.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.801+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 --> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe58c0fd260 con 0x7fe5740ac3b0 2023-10-08T09:54:48.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.801+0000 7fe5897fa700 1 -- 172.21.15.97:0/4256769783 <== osd.0 v2:172.21.15.97:6802/3104991007 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe58c0fd260 con 0x7fe5740ac3b0 2023-10-08T09:54:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] conn(0x7fe5740ac3b0 msgr2=0x7fe5740ae850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] conn(0x7fe5740ac3b0 0x7fe5740ae850 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5740a1500 msgr2=0x7fe5740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5740a1500 0x7fe5740a39c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005950 tx=0x7fe57c0058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 msgr2=0x7fe58c196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 --2- 172.21.15.97:0/4256769783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58c105690 0x7fe58c196a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe584007ee0 tx=0x7fe58400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe5909c5700 1 -- 172.21.15.97:0/4256769783 reap_dead start 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 shutdown_connections 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 >> 172.21.15.97:0/4256769783 conn(0x7fe58c0757d0 msgr2=0x7fe58c0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 shutdown_connections 2023-10-08T09:54:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.802+0000 7fe592428700 1 -- 172.21.15.97:0/4256769783 wait complete. 2023-10-08T09:54:48.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- 172.21.15.97:0/2375547643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b300ffb30 msgr2=0x7f0b3010d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 --2- 172.21.15.97:0/2375547643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b300ffb30 0x7f0b3010d5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800b0a0 tx=0x7f0b2800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- 172.21.15.97:0/2375547643 shutdown_connections 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 --2- 172.21.15.97:0/2375547643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b300ffb30 0x7f0b3010d5c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 --2- 172.21.15.97:0/2375547643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b301058d0 0x7f0b30105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 --2- 172.21.15.97:0/2375547643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b300ff190 0x7f0b300ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- 172.21.15.97:0/2375547643 >> 172.21.15.97:0/2375547643 conn(0x7f0b30074a10 msgr2=0x7f0b30074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- 172.21.15.97:0/2375547643 shutdown_connections 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- 172.21.15.97:0/2375547643 wait complete. 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 Processor -- start 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.979+0000 7f0b382fb700 1 -- start start 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b300ff190 0x7f0b30198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b300ffb30 0x7f0b301993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 0x7f0b3019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3019dd80 con 0x7f0b301058d0 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3019def0 con 0x7f0b300ffb30 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3019e0b0 con 0x7f0b300ff190 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 0x7f0b3019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 0x7f0b3019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55838/0 (socket says 172.21.15.97:55838) 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 -- 172.21.15.97:0/1958184689 learned_addr learned my addr 172.21.15.97:0/1958184689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36097700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b300ff190 0x7f0b30198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b35896700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b300ffb30 0x7f0b301993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 -- 172.21.15.97:0/1958184689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b300ff190 msgr2=0x7f0b30198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b300ff190 0x7f0b30198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 -- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b300ffb30 msgr2=0x7f0b301993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b300ffb30 0x7f0b301993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:48.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28009d00 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b36898700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 0x7f0b3019d770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800b070 tx=0x7f0b28007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800e040 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b2800be40 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280052a0 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3019e330 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.980+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b301a5bf0 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.981+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b3019e610 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.982+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0b2801d470 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b277fe700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b1c0a1480 0x7f0b1c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b280ce8b0 con 0x7f0b301058d0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b36097700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b1c0a1480 0x7f0b1c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b277fe700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] conn(0x7f0b1c0ac4a0 0x7f0b1c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b1c0af010 con 0x7f0b1c0ac4a0 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b280cecc0 con 0x7f0b301058d0 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b36097700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b1c0a1480 0x7f0b1c0a3940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b30068160 tx=0x7f0b2c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.983+0000 7f0b35896700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] conn(0x7f0b1c0ac4a0 0x7f0b1c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.984+0000 7f0b35896700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] conn(0x7f0b1c0ac4a0 0x7f0b1c0ae940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.985+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== osd.1 v2:172.21.15.178:6800/4284206053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0b1c0af010 con 0x7f0b1c0ac4a0 2023-10-08T09:54:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.002+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 --> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b301109e0 con 0x7f0b1c0ac4a0 2023-10-08T09:54:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b277fe700 1 -- 172.21.15.97:0/1958184689 <== osd.1 v2:172.21.15.178:6800/4284206053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b301109e0 con 0x7f0b1c0ac4a0 2023-10-08T09:54:49.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] conn(0x7f0b1c0ac4a0 msgr2=0x7f0b1c0ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] conn(0x7f0b1c0ac4a0 0x7f0b1c0ae940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b1c0a1480 msgr2=0x7f0b1c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b1c0a1480 0x7f0b1c0a3940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b30068160 tx=0x7f0b2c00a380 comp rx=0 tx=0).stop 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 msgr2=0x7f0b3019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 --2- 172.21.15.97:0/1958184689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b301058d0 0x7f0b3019d770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800b070 tx=0x7f0b28007930 comp rx=0 tx=0).stop 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b36898700 1 -- 172.21.15.97:0/1958184689 reap_dead start 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 shutdown_connections 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 >> 172.21.15.97:0/1958184689 conn(0x7f0b30074a10 msgr2=0x7f0b300fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 shutdown_connections 2023-10-08T09:54:49.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.003+0000 7f0b382fb700 1 -- 172.21.15.97:0/1958184689 wait complete. 2023-10-08T09:54:49.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 -- 172.21.15.97:0/601136053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92581056b0 msgr2=0x7f9258105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 --2- 172.21.15.97:0/601136053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92581056b0 0x7f9258105a90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9254009990 tx=0x7f9254009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 -- 172.21.15.97:0/601136053 shutdown_connections 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 --2- 172.21.15.97:0/601136053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9258100110 0x7f925810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 --2- 172.21.15.97:0/601136053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92581056b0 0x7f9258105a90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 --2- 172.21.15.97:0/601136053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92580ff750 0x7f92580ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.994+0000 7f926072d700 1 -- 172.21.15.97:0/601136053 >> 172.21.15.97:0/601136053 conn(0x7f9258075790 msgr2=0x7f9258075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- 172.21.15.97:0/601136053 shutdown_connections 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- 172.21.15.97:0/601136053 wait complete. 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 Processor -- start 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- start start 2023-10-08T09:54:49.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 0x7f9258195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9258100110 0x7f9258195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92581056b0 0x7f92581962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581a4f70 con 0x7f92580ff750 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581a50e0 con 0x7f92581056b0 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f926072d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581a52a0 con 0x7f9258100110 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f925e4c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 0x7f9258195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f925e4c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 0x7f9258195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55850/0 (socket says 172.21.15.97:55850) 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.995+0000 7f925e4c9700 1 -- 172.21.15.97:0/498407254 learned_addr learned my addr 172.21.15.97:0/498407254 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925ecca700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92581056b0 0x7f92581962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925dcc8700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9258100110 0x7f9258195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 -- 172.21.15.97:0/498407254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9258100110 msgr2=0x7f9258195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9258100110 0x7f9258195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 -- 172.21.15.97:0/498407254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92581056b0 msgr2=0x7f92581962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92581056b0 0x7f92581962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92540095f0 con 0x7f92580ff750 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f925e4c9700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 0x7f9258195820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92540098e0 tx=0x7f925400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925401d070 con 0x7f92580ff750 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92581a5520 con 0x7f92580ff750 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9254022470 con 0x7f92580ff750 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254017650 con 0x7f92580ff750 2023-10-08T09:54:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.996+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92581a5930 con 0x7f92580ff750 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.997+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f925810f910 con 0x7f92580ff750 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.998+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f92540225e0 con 0x7f92580ff750 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f924f7fe700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f92440a1480 0x7f92440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f925dcc8700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f92440a1480 0x7f92440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f92540ce5a0 con 0x7f92580ff750 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f924f7fe700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] conn(0x7f92440ac4a0 0x7f92440ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92440af010 con 0x7f92440ac4a0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92540ce9b0 con 0x7f92580ff750 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f925ecca700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] conn(0x7f92440ac4a0 0x7f92440ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f925dcc8700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f92440a1480 0x7f92440a3940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9258068a30 tx=0x7f924800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.999+0000 7f925ecca700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] conn(0x7f92440ac4a0 0x7f92440ae940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.000+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== osd.9 v2:172.21.15.97:6826/3522179908 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f92440af010 con 0x7f92440ac4a0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.017+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 --> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9258060ef0 con 0x7f92440ac4a0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f924f7fe700 1 -- 172.21.15.97:0/498407254 <== osd.9 v2:172.21.15.97:6826/3522179908 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9258060ef0 con 0x7f92440ac4a0 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] conn(0x7f92440ac4a0 msgr2=0x7f92440ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] conn(0x7f92440ac4a0 0x7f92440ae940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f92440a1480 msgr2=0x7f92440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f92440a1480 0x7f92440a3940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9258068a30 tx=0x7f924800a300 comp rx=0 tx=0).stop 2023-10-08T09:54:49.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 msgr2=0x7f9258195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 --2- 172.21.15.97:0/498407254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92580ff750 0x7f9258195820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92540098e0 tx=0x7f925400f670 comp rx=0 tx=0).stop 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f925ecca700 1 -- 172.21.15.97:0/498407254 reap_dead start 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 shutdown_connections 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 >> 172.21.15.97:0/498407254 conn(0x7f9258075790 msgr2=0x7f92580fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 shutdown_connections 2023-10-08T09:54:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.018+0000 7f926072d700 1 -- 172.21.15.97:0/498407254 wait complete. 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 -- 172.21.15.97:0/2819295863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf00ff750 msgr2=0x7fbaf00ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/2819295863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf00ff750 0x7fbaf00ffbd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009990 tx=0x7fbae0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 -- 172.21.15.97:0/2819295863 shutdown_connections 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/2819295863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0100110 0x7fbaf010c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/2819295863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaf0105690 0x7fbaf0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/2819295863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf00ff750 0x7fbaf00ffbd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 -- 172.21.15.97:0/2819295863 >> 172.21.15.97:0/2819295863 conn(0x7fbaf00757d0 msgr2=0x7fbaf0075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 -- 172.21.15.97:0/2819295863 shutdown_connections 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.632+0000 7fbaf7a12700 1 -- 172.21.15.97:0/2819295863 wait complete. 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.633+0000 7fbaf7a12700 1 Processor -- start 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.633+0000 7fbaf7a12700 1 -- start start 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaf00ff750 0x7fbaf01979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0100110 0x7fbaf0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 0x7fbaf01a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf0198800 con 0x7fbaf0105690 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf01a2f90 con 0x7fbaf00ff750 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf7a12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf01a3100 con 0x7fbaf0100110 2023-10-08T09:54:49.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 0x7fbaf01a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 0x7fbaf01a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55790/0 (socket says 172.21.15.97:55790) 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 -- 172.21.15.97:0/3561436901 learned_addr learned my addr 172.21.15.97:0/3561436901 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf4fad700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0100110 0x7fbaf0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf57ae700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaf00ff750 0x7fbaf01979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 -- 172.21.15.97:0/3561436901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0100110 msgr2=0x7fbaf0197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0100110 0x7fbaf0197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 -- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaf00ff750 msgr2=0x7fbaf01979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbaf00ff750 0x7fbaf01979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00095f0 con 0x7fbaf0105690 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.634+0000 7fbaf5faf700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 0x7fbaf01a2a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbae8007ee0 tx=0x7fbae800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8017070 con 0x7fbaf0105690 2023-10-08T09:54:49.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf01a33e0 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae800f040 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8005740 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf01a38b0 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.635+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbaf010f8f0 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.637+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbae800dc60 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbae67fc700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbadc0a1480 0x7fbadc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbaf57ae700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbadc0a1480 0x7fbadc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbae80ccee0 con 0x7fbaf0105690 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbae67fc700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] conn(0x7fbadc0ac4a0 0x7fbadc0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbadc0af010 con 0x7fbadc0ac4a0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae80cd250 con 0x7fbaf0105690 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.638+0000 7fbaf57ae700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbadc0a1480 0x7fbadc0a3940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005950 tx=0x7fbaec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.639+0000 7fbaf4fad700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] conn(0x7fbadc0ac4a0 0x7fbadc0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.639+0000 7fbaf4fad700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] conn(0x7fbadc0ac4a0 0x7fbadc0ae940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.640+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== osd.7 v2:172.21.15.178:6816/2550553059 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbadc0af010 con 0x7fbadc0ac4a0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.668+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 --> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbaf0100dd0 con 0x7fbadc0ac4a0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.669+0000 7fbae67fc700 1 -- 172.21.15.97:0/3561436901 <== osd.7 v2:172.21.15.178:6816/2550553059 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbaf0100dd0 con 0x7fbadc0ac4a0 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] conn(0x7fbadc0ac4a0 msgr2=0x7fbadc0ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] conn(0x7fbadc0ac4a0 0x7fbadc0ae940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbadc0a1480 msgr2=0x7fbadc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbadc0a1480 0x7fbadc0a3940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005950 tx=0x7fbaec0058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 msgr2=0x7fbaf01a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 --2- 172.21.15.97:0/3561436901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf0105690 0x7fbaf01a2a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbae8007ee0 tx=0x7fbae800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf5faf700 1 -- 172.21.15.97:0/3561436901 reap_dead start 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 shutdown_connections 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 >> 172.21.15.97:0/3561436901 conn(0x7fbaf00757d0 msgr2=0x7fbaf00fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 shutdown_connections 2023-10-08T09:54:49.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.670+0000 7fbaf7a12700 1 -- 172.21.15.97:0/3561436901 wait complete. 2023-10-08T09:54:49.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:49 smithi097 ceph-mon[99491]: pgmap v288: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:49 smithi186 ceph-mon[104393]: pgmap v288: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:49 smithi178 ceph-mon[103156]: pgmap v288: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:49.764 INFO:teuthology.orchestra.run.smithi097.stdout:210453397541 2023-10-08T09:54:49.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-08T09:54:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 -- 172.21.15.97:0/324175061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ff150 msgr2=0x7f13780ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 --2- 172.21.15.97:0/324175061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ff150 0x7f13780ff5d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f136c009990 tx=0x7f136c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 -- 172.21.15.97:0/324175061 shutdown_connections 2023-10-08T09:54:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 --2- 172.21.15.97:0/324175061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13780ffb10 0x7f137810d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 --2- 172.21.15.97:0/324175061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 0x7f1378105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 --2- 172.21.15.97:0/324175061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ff150 0x7f13780ff5d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 -- 172.21.15.97:0/324175061 >> 172.21.15.97:0/324175061 conn(0x7f1378074aa0 msgr2=0x7f1378074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.684+0000 7f137e5aa700 1 -- 172.21.15.97:0/324175061 shutdown_connections 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.685+0000 7f137e5aa700 1 -- 172.21.15.97:0/324175061 wait complete. 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.685+0000 7f137e5aa700 1 Processor -- start 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.685+0000 7f137e5aa700 1 -- start start 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13780ff150 0x7f137819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 0x7f137819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 0x7f1378196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13781970e0 con 0x7f13780ffb10 2023-10-08T09:54:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378197250 con 0x7f13780ff150 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137e5aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378197410 con 0x7f1378105840 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 0x7f137819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 0x7f137819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55804/0 (socket says 172.21.15.97:55804) 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 -- 172.21.15.97:0/3418000181 learned_addr learned my addr 172.21.15.97:0/3418000181 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f1377fff700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13780ff150 0x7f137819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137cb47700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 0x7f1378196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 -- 172.21.15.97:0/3418000181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 msgr2=0x7f1378196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 0x7f1378196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 -- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13780ff150 msgr2=0x7f137819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13780ff150 0x7f137819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f13777fe700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0095f0 con 0x7f13780ffb10 2023-10-08T09:54:49.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.686+0000 7f137cb47700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1378105840 0x7f1378196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f13777fe700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 0x7f137819ce70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f136c00f6a0 tx=0x7f136c00f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c01d070 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136c00fbf0 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1378197690 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c017650 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13781a3920 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.687+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1378110a50 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.689+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f136c005080 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.689+0000 7f13757fa700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f13600a1500 0x7f13600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f1377fff700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f13600a1500 0x7f13600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f136c0ceb90 con 0x7f13780ffb10 2023-10-08T09:54:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f13757fa700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] conn(0x7f13600ac4a0 0x7f13600ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13600af010 con 0x7f13600ac4a0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f136c0cef40 con 0x7f13780ffb10 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f1377fff700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f13600a1500 0x7f13600a39c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f136800abd0 tx=0x7f136800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f137cb47700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] conn(0x7f13600ac4a0 0x7f13600ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.690+0000 7f137cb47700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] conn(0x7f13600ac4a0 0x7f13600ae940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.691+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== osd.4 v2:172.21.15.178:6808/1866336071 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f13600af010 con 0x7f13600ac4a0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.709+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 --> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1378060ef0 con 0x7f13600ac4a0 2023-10-08T09:54:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f13757fa700 1 -- 172.21.15.97:0/3418000181 <== osd.4 v2:172.21.15.178:6808/1866336071 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1378060ef0 con 0x7f13600ac4a0 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] conn(0x7f13600ac4a0 msgr2=0x7f13600ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] conn(0x7f13600ac4a0 0x7f13600ae940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f13600a1500 msgr2=0x7f13600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f13600a1500 0x7f13600a39c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f136800abd0 tx=0x7f136800a380 comp rx=0 tx=0).stop 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 msgr2=0x7f137819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137e5aa700 1 --2- 172.21.15.97:0/3418000181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13780ffb10 0x7f137819ce70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f136c00f6a0 tx=0x7f136c00f6d0 comp rx=0 tx=0).stop 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.710+0000 7f137cb47700 1 -- 172.21.15.97:0/3418000181 reap_dead start 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.711+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 shutdown_connections 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.711+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 >> 172.21.15.97:0/3418000181 conn(0x7f1378074aa0 msgr2=0x7f13780fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.711+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 shutdown_connections 2023-10-08T09:54:49.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:48.711+0000 7f137e5aa700 1 -- 172.21.15.97:0/3418000181 wait complete. 2023-10-08T09:54:50.402 INFO:teuthology.orchestra.run.smithi097.stdout:240518168609 2023-10-08T09:54:50.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-08T09:54:50.638 INFO:teuthology.orchestra.run.smithi097.stdout:34359738435 2023-10-08T09:54:50.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-08T09:54:50.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 -- 172.21.15.97:0/1104831966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d80ff190 msgr2=0x7fb4d80ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 --2- 172.21.15.97:0/1104831966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d80ff190 0x7fb4d80ff5f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009990 tx=0x7fb4c8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 -- 172.21.15.97:0/1104831966 shutdown_connections 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 --2- 172.21.15.97:0/1104831966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d80ffb30 0x7fb4d810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 --2- 172.21.15.97:0/1104831966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d81058d0 0x7fb4d8105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 --2- 172.21.15.97:0/1104831966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d80ff190 0x7fb4d80ff5f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 -- 172.21.15.97:0/1104831966 >> 172.21.15.97:0/1104831966 conn(0x7fb4d8074a10 msgr2=0x7fb4d8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 -- 172.21.15.97:0/1104831966 shutdown_connections 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.950+0000 7fb4e056a700 1 -- 172.21.15.97:0/1104831966 wait complete. 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 Processor -- start 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 -- start start 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d80ff190 0x7fb4d819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d80ffb30 0x7fb4d819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 0x7fb4d8196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8196ff0 con 0x7fb4d81058d0 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8197160 con 0x7fb4d80ff190 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4e056a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8197320 con 0x7fb4d80ffb30 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 0x7fb4d8196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 0x7fb4d8196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55870/0 (socket says 172.21.15.97:55870) 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 -- 172.21.15.97:0/278295310 learned_addr learned my addr 172.21.15.97:0/278295310 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4ddb05700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d80ffb30 0x7fb4d819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4de306700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d80ff190 0x7fb4d819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 -- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d80ffb30 msgr2=0x7fb4d819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4d80ffb30 0x7fb4d819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 -- 172.21.15.97:0/278295310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d80ff190 msgr2=0x7fb4d819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4d80ff190 0x7fb4d819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.951+0000 7fb4deb07700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80095f0 con 0x7fb4d81058d0 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4deb07700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 0x7fb4d8196980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0007d60 tx=0x7fb4d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000ad40 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d0004030 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d8197600 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000aea0 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d81a3830 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.952+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4d81109e0 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb4d00041a0 con 0x7fb4d81058d0 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4c40a1480 0x7fb4c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:50.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4de306700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4c40a1480 0x7fb4c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb4d00cd9b0 con 0x7fb4d81058d0 2023-10-08T09:54:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] conn(0x7fb4c40ac4a0 0x7fb4c40ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4c40af010 con 0x7fb4c40ac4a0 2023-10-08T09:54:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.955+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d00cddc0 con 0x7fb4d81058d0 2023-10-08T09:54:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.956+0000 7fb4de306700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4c40a1480 0x7fb4c40a3940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005950 tx=0x7fb4d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.956+0000 7fb4ddb05700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] conn(0x7fb4c40ac4a0 0x7fb4c40ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.956+0000 7fb4ddb05700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] conn(0x7fb4c40ac4a0 0x7fb4c40ae940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.957+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== osd.2 v2:172.21.15.186:6800/1978590862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb4c40af010 con 0x7fb4c40ac4a0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.973+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 --> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4d8060ef0 con 0x7fb4c40ac4a0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4cf7fe700 1 -- 172.21.15.97:0/278295310 <== osd.2 v2:172.21.15.186:6800/1978590862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4d8060ef0 con 0x7fb4c40ac4a0 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] conn(0x7fb4c40ac4a0 msgr2=0x7fb4c40ae940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] conn(0x7fb4c40ac4a0 0x7fb4c40ae940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4c40a1480 msgr2=0x7fb4c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb4c40a1480 0x7fb4c40a3940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005950 tx=0x7fb4d40058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 msgr2=0x7fb4d8196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 --2- 172.21.15.97:0/278295310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4d81058d0 0x7fb4d8196980 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0007d60 tx=0x7fb4d000a340 comp rx=0 tx=0).stop 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4deb07700 1 -- 172.21.15.97:0/278295310 reap_dead start 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 shutdown_connections 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 >> 172.21.15.97:0/278295310 conn(0x7fb4d8074a10 msgr2=0x7fb4d80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 shutdown_connections 2023-10-08T09:54:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:49.974+0000 7fb4e056a700 1 -- 172.21.15.97:0/278295310 wait complete. 2023-10-08T09:54:50.897 INFO:teuthology.orchestra.run.smithi097.stdout:270582939678 2023-10-08T09:54:50.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-08T09:54:50.992 INFO:teuthology.orchestra.run.smithi097.stdout:55834574910 2023-10-08T09:54:50.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-08T09:54:51.152 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:51 smithi097 ceph-mon[99491]: pgmap v289: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:51 smithi178 ceph-mon[103156]: pgmap v289: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:51.476 INFO:teuthology.orchestra.run.smithi097.stdout:137438953522 2023-10-08T09:54:51.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-08T09:54:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:51 smithi186 ceph-mon[104393]: pgmap v289: 129 pgs: 5 peering, 124 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 0 objects/s recovering 2023-10-08T09:54:51.782 INFO:teuthology.orchestra.run.smithi097.stdout:77309411387 2023-10-08T09:54:51.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- 172.21.15.97:0/3764997814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c104680 msgr2=0x7f1c8c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/3764997814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c104680 0x7f1c8c104a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1c800099c0 tx=0x7f1c80009cd0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- 172.21.15.97:0/3764997814 shutdown_connections 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/3764997814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c8c1003b0 0x7f1c8c10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/3764997814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c104680 0x7f1c8c104a60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/3764997814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- 172.21.15.97:0/3764997814 >> 172.21.15.97:0/3764997814 conn(0x7f1c8c076510 msgr2=0x7f1c8c076920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- 172.21.15.97:0/3764997814 shutdown_connections 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- 172.21.15.97:0/3764997814 wait complete. 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 Processor -- start 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.947+0000 7f1c92bfa700 1 -- start start 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c8c1003b0 0x7f1c8c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c8c104680 0x7f1c8c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c197020 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c197190 con 0x7f1c8c104680 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c92bfa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c197350 con 0x7f1c8c1003b0 2023-10-08T09:54:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55880/0 (socket says 172.21.15.97:55880) 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 -- 172.21.15.97:0/46876405 learned_addr learned my addr 172.21.15.97:0/46876405 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c91197700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c8c104680 0x7f1c8c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c8bfff700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c8c1003b0 0x7f1c8c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 -- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c8c1003b0 msgr2=0x7f1c8c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c8c1003b0 0x7f1c8c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 -- 172.21.15.97:0/46876405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c8c104680 msgr2=0x7f1c8c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c8c104680 0x7f1c8c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800095f0 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.948+0000 7f1c90996700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c19c880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c800098e0 tx=0x7f1c800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.949+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001d070 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.949+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8000fbb0 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.949+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80022d40 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.949+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c1975d0 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.949+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c1a3870 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.950+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c8c10f900 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.952+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1c8000fd20 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.952+0000 7f1c89ffb700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1c7c0a15d0 0x7f1c7c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c800cecf0 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c8bfff700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1c7c0a15d0 0x7f1c7c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c89ffb700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] conn(0x7f1c7c0ac5f0 0x7f1c7c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c7c0af160 con 0x7f1c7c0ac5f0 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c80022ea0 con 0x7f1c8c0ff9f0 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c91197700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] conn(0x7f1c7c0ac5f0 0x7f1c7c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:51.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.953+0000 7f1c8bfff700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1c7c0a15d0 0x7f1c7c0a3a90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005fd0 tx=0x7f1c78005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:51.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.954+0000 7f1c91197700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] conn(0x7f1c7c0ac5f0 0x7f1c7c0aea90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:51.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.954+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== osd.11 v2:172.21.15.186:6824/1987590926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1c7c0af160 con 0x7f1c7c0ac5f0 2023-10-08T09:54:51.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.971+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 --> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c8c103a40 con 0x7f1c7c0ac5f0 2023-10-08T09:54:51.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.972+0000 7f1c89ffb700 1 -- 172.21.15.97:0/46876405 <== osd.11 v2:172.21.15.186:6824/1987590926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c8c103a40 con 0x7f1c7c0ac5f0 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] conn(0x7f1c7c0ac5f0 msgr2=0x7f1c7c0aea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] conn(0x7f1c7c0ac5f0 0x7f1c7c0aea90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1c7c0a15d0 msgr2=0x7f1c7c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1c7c0a15d0 0x7f1c7c0a3a90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005fd0 tx=0x7f1c78005f60 comp rx=0 tx=0).stop 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 msgr2=0x7f1c8c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 --2- 172.21.15.97:0/46876405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c8c0ff9f0 0x7f1c8c19c880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c800098e0 tx=0x7f1c800057f0 comp rx=0 tx=0).stop 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c91197700 1 -- 172.21.15.97:0/46876405 reap_dead start 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 shutdown_connections 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 >> 172.21.15.97:0/46876405 conn(0x7f1c8c076510 msgr2=0x7f1c8c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 shutdown_connections 2023-10-08T09:54:51.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:51.973+0000 7f1c92bfa700 1 -- 172.21.15.97:0/46876405 wait complete. 2023-10-08T09:54:52.283 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:52.619 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:52.654 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:53.258 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:53 smithi097 ceph-mon[99491]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 494 B/s, 1 objects/s recovering 2023-10-08T09:54:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:53 smithi178 ceph-mon[103156]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 494 B/s, 1 objects/s recovering 2023-10-08T09:54:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:53.515 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:53.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:53 smithi186 ceph-mon[104393]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 494 B/s, 1 objects/s recovering 2023-10-08T09:54:54.422 INFO:teuthology.orchestra.run.smithi097.stdout:326417514518 2023-10-08T09:54:54.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-08T09:54:55.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:55 smithi097 ceph-mon[99491]: pgmap v291: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 499 B/s, 1 objects/s recovering 2023-10-08T09:54:55.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:55 smithi178 ceph-mon[103156]: pgmap v291: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 499 B/s, 1 objects/s recovering 2023-10-08T09:54:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:55 smithi186 ceph-mon[104393]: pgmap v291: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 499 B/s, 1 objects/s recovering 2023-10-08T09:54:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 -- 172.21.15.97:0/2023628746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc104720 msgr2=0x7ffacc104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 --2- 172.21.15.97:0/2023628746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc104720 0x7ffacc104b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009990 tx=0x7ffac0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 -- 172.21.15.97:0/2023628746 shutdown_connections 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 --2- 172.21.15.97:0/2023628746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc100c60 0x7ffacc10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 --2- 172.21.15.97:0/2023628746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc104720 0x7ffacc104b00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 --2- 172.21.15.97:0/2023628746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc1002a0 0x7ffacc100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 -- 172.21.15.97:0/2023628746 >> 172.21.15.97:0/2023628746 conn(0x7ffacc074a10 msgr2=0x7ffacc074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 -- 172.21.15.97:0/2023628746 shutdown_connections 2023-10-08T09:54:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 -- 172.21.15.97:0/2023628746 wait complete. 2023-10-08T09:54:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.060+0000 7ffad1cad700 1 Processor -- start 2023-10-08T09:54:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 -- start start 2023-10-08T09:54:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc100c60 0x7ffacc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 0x7ffacc1a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc198900 con 0x7ffacc1002a0 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc1a30c0 con 0x7ffacc104720 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc1a3200 con 0x7ffacc100c60 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55894/0 (socket says 172.21.15.97:55894) 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 -- 172.21.15.97:0/719887646 learned_addr learned my addr 172.21.15.97:0/719887646 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacbfff700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 0x7ffacc1a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacaffd700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc100c60 0x7ffacc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 -- 172.21.15.97:0/719887646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc100c60 msgr2=0x7ffacc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc100c60 0x7ffacc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 -- 172.21.15.97:0/719887646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 msgr2=0x7ffacc1a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 0x7ffacc1a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 -- 172.21.15.97:0/719887646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00095f0 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacbfff700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 0x7ffacc1a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffacb7fe700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffac00098e0 tx=0x7ffac000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac001d070 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac000fb90 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc1958f0 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.061+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0017650 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.062+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc195d90 con 0x7ffacc1002a0 2023-10-08T09:54:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.062+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc107c80 con 0x7ffacc1002a0 2023-10-08T09:54:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.064+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ffac000fd00 con 0x7ffacc1002a0 2023-10-08T09:54:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.064+0000 7ffac8ff9700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 0x7ffab40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.064+0000 7ffacaffd700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 0x7ffab40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.064+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffac00ce4e0 con 0x7ffacc1002a0 2023-10-08T09:54:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.065+0000 7ffacaffd700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 0x7ffab40a3940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffabc005950 tx=0x7ffabc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.068+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffac00978c0 con 0x7ffacc1002a0 2023-10-08T09:54:56.171 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:56.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.271+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffacc04e4c0 con 0x7ffacc1002a0 2023-10-08T09:54:56.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.272+0000 7ffac8ff9700 1 -- 172.21.15.97:0/719887646 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffac009aee0 con 0x7ffacc1002a0 2023-10-08T09:54:56.274 INFO:teuthology.orchestra.run.smithi097.stdout:34359738436 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 msgr2=0x7ffab40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 0x7ffab40a3940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffabc005950 tx=0x7ffabc0058e0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 msgr2=0x7ffacc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffac00098e0 tx=0x7ffac000f720 comp rx=0 tx=0).stop 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 shutdown_connections 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ffab40a1480 0x7ffab40a3940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffacc104720 0x7ffacc1a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffacc1002a0 0x7ffacc197b00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 --2- 172.21.15.97:0/719887646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffacc100c60 0x7ffacc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 >> 172.21.15.97:0/719887646 conn(0x7ffacc074a10 msgr2=0x7ffacc0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 shutdown_connections 2023-10-08T09:54:56.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:56.274+0000 7ffad1cad700 1 -- 172.21.15.97:0/719887646 wait complete. 2023-10-08T09:54:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:57.165 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:54:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2394187981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068190 msgr2=0x7f46b40685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2394187981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068190 0x7f46b40685f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f469c009990 tx=0x7f469c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2394187981 shutdown_connections 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2394187981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068b30 0x7f46b410d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2394187981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 0x7f46b4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2394187981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068190 0x7f46b40685f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2394187981 >> 172.21.15.97:0/2394187981 conn(0x7f46b4075790 msgr2=0x7f46b4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.281+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2394187981 shutdown_connections 2023-10-08T09:54:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.282+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2394187981 wait complete. 2023-10-08T09:54:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.282+0000 7f46b8dc9700 1 Processor -- start 2023-10-08T09:54:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.282+0000 7f46b8dc9700 1 -- start start 2023-10-08T09:54:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068190 0x7f46b419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 0x7f46b4196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41970e0 con 0x7f46b4068b30 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4197250 con 0x7f46b41067d0 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4197410 con 0x7f46b4068190 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55910/0 (socket says 172.21.15.97:55910) 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 -- 172.21.15.97:0/2626217609 learned_addr learned my addr 172.21.15.97:0/2626217609 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b259c700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068190 0x7f46b419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b2d9d700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 0x7f46b4196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 -- 172.21.15.97:0/2626217609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068190 msgr2=0x7f46b419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068190 0x7f46b419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 -- 172.21.15.97:0/2626217609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 msgr2=0x7f46b4196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 0x7f46b4196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 -- 172.21.15.97:0/2626217609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0095f0 con 0x7f46b4068b30 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b1d9b700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f469c000c00 tx=0x7f469c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c01d070 con 0x7f46b4068b30 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c022470 con 0x7f46b4068b30 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c017650 con 0x7f46b4068b30 2023-10-08T09:54:57.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4197690 con 0x7f46b4068b30 2023-10-08T09:54:57.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.283+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41a3930 con 0x7f46b4068b30 2023-10-08T09:54:57.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.284+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4108d80 con 0x7f46b4068b30 2023-10-08T09:54:57.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.285+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f469c0225e0 con 0x7f46b4068b30 2023-10-08T09:54:57.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.286+0000 7f46ab7fe700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 0x7f46a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:57.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.286+0000 7f46b259c700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 0x7f46a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:57.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.286+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f469c0ce890 con 0x7f46b4068b30 2023-10-08T09:54:57.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.287+0000 7f46b259c700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 0x7f46a00a3940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f540 tx=0x7f46ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:57.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.289+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f469c097d90 con 0x7f46b4068b30 2023-10-08T09:54:57.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:57 smithi097 ceph-mon[99491]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 413 B/s, 1 objects/s recovering 2023-10-08T09:54:57.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/719887646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:54:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:57 smithi178 ceph-mon[103156]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 413 B/s, 1 objects/s recovering 2023-10-08T09:54:57.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/719887646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:54:57.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.486+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f46b4197f40 con 0x7f46b4068b30 2023-10-08T09:54:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.487+0000 7f46ab7fe700 1 -- 172.21.15.97:0/2626217609 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f469c09b3b0 con 0x7f46b4068b30 2023-10-08T09:54:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:210453397543 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 msgr2=0x7f46a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 0x7f46a00a3940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00f540 tx=0x7f46ac00b040 comp rx=0 tx=0).stop 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 msgr2=0x7f46b419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f469c000c00 tx=0x7f469c004730 comp rx=0 tx=0).stop 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 shutdown_connections 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f46a00a1480 0x7f46a00a3940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41067d0 0x7f46b4196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f46b4068190 0x7f46b419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 --2- 172.21.15.97:0/2626217609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4068b30 0x7f46b419ce80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 >> 172.21.15.97:0/2626217609 conn(0x7f46b4075790 msgr2=0x7f46b40fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 shutdown_connections 2023-10-08T09:54:57.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:57.489+0000 7f46b8dc9700 1 -- 172.21.15.97:0/2626217609 wait complete. 2023-10-08T09:54:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:57 smithi186 ceph-mon[104393]: pgmap v292: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 413 B/s, 1 objects/s recovering 2023-10-08T09:54:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/719887646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:54:57.769 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738435 got 34359738436 for osd.0 2023-10-08T09:54:57.769 DEBUG:teuthology.parallel:result is None 2023-10-08T09:54:58.353 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2626217609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:54:58.354 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:58 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:54:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2626217609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:54:58.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:58 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.504+0000 7f2244ffa700 1 -- 172.21.15.97:0/2103327541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100110 msgr2=0x7f224010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.504+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2103327541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100110 0x7f224010c580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f223800b0a0 tx=0x7f223800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 -- 172.21.15.97:0/2103327541 shutdown_connections 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2103327541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100110 0x7f224010c580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2103327541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22401056b0 0x7f2240105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2103327541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22400ff750 0x7f22400ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 -- 172.21.15.97:0/2103327541 >> 172.21.15.97:0/2103327541 conn(0x7f2240075790 msgr2=0x7f2240075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 -- 172.21.15.97:0/2103327541 shutdown_connections 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 -- 172.21.15.97:0/2103327541 wait complete. 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 Processor -- start 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.505+0000 7f2244ffa700 1 -- start start 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22400ff750 0x7f2240197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240198960 con 0x7f22401056b0 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a3120 con 0x7f2240100110 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f2244ffa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a3260 con 0x7f22400ff750 2023-10-08T09:54:58.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22400ff750 0x7f2240197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:35892/0 (socket says 172.21.15.97:35892) 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 -- 172.21.15.97:0/2072133025 learned_addr learned my addr 172.21.15.97:0/2072133025 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55924/0 (socket says 172.21.15.97:55924) 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 -- 172.21.15.97:0/2072133025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22400ff750 msgr2=0x7f2240197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22400ff750 0x7f2240197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 -- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 msgr2=0x7f22401a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 -- 172.21.15.97:0/2072133025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009d00 con 0x7f2240100110 2023-10-08T09:54:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223ed9d700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.506+0000 7f223dd9b700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f223400e9a0 tx=0x7f223400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.507+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234009790 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.507+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2240195a10 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.507+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2240195f60 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.507+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2234010430 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.507+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234018680 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.508+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2240107cb0 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.509+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f22340187e0 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.509+0000 7f222f7fe700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 0x7f22280a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.509+0000 7f223e59c700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 0x7f22280a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.509+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2234014070 con 0x7f2240100110 2023-10-08T09:54:58.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.510+0000 7f223e59c700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 0x7f22280a39c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2230005950 tx=0x7f223000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.514+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2234052100 con 0x7f2240100110 2023-10-08T09:54:58.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2626217609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:54:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:58 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 -- 172.21.15.97:0/451809536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701057c0 msgr2=0x7fd870105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 --2- 172.21.15.97:0/451809536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701057c0 0x7fd870105ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd860009990 tx=0x7fd860009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 -- 172.21.15.97:0/451809536 shutdown_connections 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 --2- 172.21.15.97:0/451809536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd87010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 --2- 172.21.15.97:0/451809536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701057c0 0x7fd870105ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 --2- 172.21.15.97:0/451809536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701008a0 0x7fd870100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 -- 172.21.15.97:0/451809536 >> 172.21.15.97:0/451809536 conn(0x7fd8700779e0 msgr2=0x7fd870077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 -- 172.21.15.97:0/451809536 shutdown_connections 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.553+0000 7fd876488700 1 -- 172.21.15.97:0/451809536 wait complete. 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.554+0000 7fd876488700 1 Processor -- start 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.554+0000 7fd876488700 1 -- start start 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd870073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701057c0 0x7fd870073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701a4fa0 con 0x7fd8701008a0 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701a5110 con 0x7fd8701057c0 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd876488700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701a52d0 con 0x7fd870101260 2023-10-08T09:54:58.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55934/0 (socket says 172.21.15.97:55934) 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86ffff700 1 -- 172.21.15.97:0/519780928 learned_addr learned my addr 172.21.15.97:0/519780928 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd870073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd874a25700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701057c0 0x7fd870073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 -- 172.21.15.97:0/519780928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701057c0 msgr2=0x7fd870073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701057c0 0x7fd870073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 -- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 msgr2=0x7fd870072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 -- 172.21.15.97:0/519780928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600095f0 con 0x7fd870101260 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86ffff700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.555+0000 7fd86f7fe700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd870073030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d750 tx=0x7fd86400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.556+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864004020 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.556+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd864004830 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.556+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8701a55b0 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.556+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640054c0 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.556+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8701a5ab0 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.557+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870108d80 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.558+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd8640049a0 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.559+0000 7fd86d7fa700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 0x7fd8580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.559+0000 7fd86ffff700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 0x7fd8580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.559+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd8640cc8f0 con 0x7fd870101260 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.560+0000 7fd86ffff700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 0x7fd8580a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b5c0 tx=0x7fd86001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.564+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd864095c40 con 0x7fd870101260 2023-10-08T09:54:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 -- 172.21.15.97:0/2686178459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e41057c0 msgr2=0x7f10e4105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 --2- 172.21.15.97:0/2686178459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e41057c0 0x7f10e4105ba0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009990 tx=0x7f10d8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 -- 172.21.15.97:0/2686178459 shutdown_connections 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 --2- 172.21.15.97:0/2686178459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e4101260 0x7f10e410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 --2- 172.21.15.97:0/2686178459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e41057c0 0x7f10e4105ba0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 --2- 172.21.15.97:0/2686178459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41008a0 0x7f10e4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 -- 172.21.15.97:0/2686178459 >> 172.21.15.97:0/2686178459 conn(0x7f10e40779e0 msgr2=0x7f10e4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 -- 172.21.15.97:0/2686178459 shutdown_connections 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.663+0000 7f10eac58700 1 -- 172.21.15.97:0/2686178459 wait complete. 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 Processor -- start 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 -- start start 2023-10-08T09:54:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e41008a0 0x7f10e419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41057c0 0x7f10e4196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4197230 con 0x7f10e4101260 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e41973a0 con 0x7f10e41008a0 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10eac58700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4197560 con 0x7f10e41057c0 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55948/0 (socket says 172.21.15.97:55948) 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10e3fff700 1 -- 172.21.15.97:0/1448250753 learned_addr learned my addr 172.21.15.97:0/1448250753 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10e89f4700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e41008a0 0x7f10e419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.664+0000 7f10e91f5700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41057c0 0x7f10e4196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 -- 172.21.15.97:0/1448250753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41057c0 msgr2=0x7f10e4196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41057c0 0x7f10e4196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 -- 172.21.15.97:0/1448250753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e41008a0 msgr2=0x7f10e419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e41008a0 0x7f10e419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 -- 172.21.15.97:0/1448250753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc009c40 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e3fff700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9c0 tx=0x7f10d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00097b0 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10d000ee60 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d80095f0 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00054f0 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.665+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e41a3950 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.666+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e4108d80 con 0x7f10e4101260 2023-10-08T09:54:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.667+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f10d0004750 con 0x7f10e4101260 2023-10-08T09:54:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.668+0000 7f10e1ffb700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 0x7f10d40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.668+0000 7f10e89f4700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 0x7f10d40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.668+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10d0014070 con 0x7f10e4101260 2023-10-08T09:54:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.668+0000 7f10e89f4700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 0x7f10d40a39c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.672+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f10d00964e0 con 0x7f10e4101260 2023-10-08T09:54:58.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.720+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f224004e4f0 con 0x7f2240100110 2023-10-08T09:54:58.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.721+0000 7f222f7fe700 1 -- 172.21.15.97:0/2072133025 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22340964f0 con 0x7f2240100110 2023-10-08T09:54:58.723 INFO:teuthology.orchestra.run.smithi097.stdout:188978561068 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 msgr2=0x7f22280a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 0x7f22280a39c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2230005950 tx=0x7f223000b410 comp rx=0 tx=0).stop 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 msgr2=0x7f22401980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f223400e9a0 tx=0x7f223400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 shutdown_connections 2023-10-08T09:54:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f22280a1500 0x7f22280a39c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22401056b0 0x7f22401a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22400ff750 0x7f2240197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 --2- 172.21.15.97:0/2072133025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2240100110 0x7f22401980a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 >> 172.21.15.97:0/2072133025 conn(0x7f2240075790 msgr2=0x7f22400fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 shutdown_connections 2023-10-08T09:54:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.723+0000 7f2244ffa700 1 -- 172.21.15.97:0/2072133025 wait complete. 2023-10-08T09:54:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.766+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd870073fd0 con 0x7fd870101260 2023-10-08T09:54:58.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.767+0000 7fd86d7fa700 1 -- 172.21.15.97:0/519780928 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd864099260 con 0x7fd870101260 2023-10-08T09:54:58.769 INFO:teuthology.orchestra.run.smithi097.stdout:240518168611 2023-10-08T09:54:58.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 msgr2=0x7fd8580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 0x7fd8580a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd86000b5c0 tx=0x7fd86001a040 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 msgr2=0x7fd870073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd870073030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd86400d750 tx=0x7fd86400da60 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 shutdown_connections 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8580a1480 0x7fd8580a3940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8701057c0 0x7fd870073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8701008a0 0x7fd870072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 --2- 172.21.15.97:0/519780928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd870101260 0x7fd870073030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 >> 172.21.15.97:0/519780928 conn(0x7fd8700779e0 msgr2=0x7fd8700fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 shutdown_connections 2023-10-08T09:54:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.769+0000 7fd876488700 1 -- 172.21.15.97:0/519780928 wait complete. 2023-10-08T09:54:58.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.885+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10e4197f60 con 0x7f10e4101260 2023-10-08T09:54:58.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.886+0000 7f10e1ffb700 1 -- 172.21.15.97:0/1448250753 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10d0099b00 con 0x7f10e4101260 2023-10-08T09:54:58.888 INFO:teuthology.orchestra.run.smithi097.stdout:107374182455 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 msgr2=0x7f10d40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 0x7f10d40a39c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d8005960 comp rx=0 tx=0).stop 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 msgr2=0x7f10e419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d000e9c0 tx=0x7f10d000ecd0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 shutdown_connections 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10d40a1500 0x7f10d40a39c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10e41057c0 0x7f10e4196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e41008a0 0x7f10e419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 --2- 172.21.15.97:0/1448250753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e4101260 0x7f10e419cf10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:58.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 >> 172.21.15.97:0/1448250753 conn(0x7f10e40779e0 msgr2=0x7f10e40fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:58.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 shutdown_connections 2023-10-08T09:54:58.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:58.888+0000 7f10eac58700 1 -- 172.21.15.97:0/1448250753 wait complete. 2023-10-08T09:54:59.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397541 got 210453397543 for osd.7 2023-10-08T09:54:59.236 DEBUG:teuthology.parallel:result is None 2023-10-08T09:54:59.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:59 smithi097 ceph-mon[99491]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 375 B/s, 1 objects/s recovering 2023-10-08T09:54:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2072133025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:54:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/519780928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:54:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:54:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1448250753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:54:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:59 smithi178 ceph-mon[103156]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 375 B/s, 1 objects/s recovering 2023-10-08T09:54:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2072133025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:54:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/519780928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:54:59.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:54:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1448250753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:54:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:59 smithi186 ceph-mon[104393]: pgmap v293: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 375 B/s, 1 objects/s recovering 2023-10-08T09:54:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2072133025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:54:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/519780928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:54:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:54:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1448250753' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.864+0000 7f82d11c2700 1 -- 172.21.15.97:0/1404687345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc068190 msgr2=0x7f82cc0685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.864+0000 7f82d11c2700 1 --2- 172.21.15.97:0/1404687345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc068190 0x7f82cc0685f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009990 tx=0x7f82c0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.864+0000 7f82d11c2700 1 -- 172.21.15.97:0/1404687345 shutdown_connections 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.864+0000 7f82d11c2700 1 --2- 172.21.15.97:0/1404687345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 0x7f82cc10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.864+0000 7f82d11c2700 1 --2- 172.21.15.97:0/1404687345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc1067a0 0x7f82cc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.865+0000 7f82d11c2700 1 --2- 172.21.15.97:0/1404687345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc068190 0x7f82cc0685f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.865+0000 7f82d11c2700 1 -- 172.21.15.97:0/1404687345 >> 172.21.15.97:0/1404687345 conn(0x7f82cc075790 msgr2=0x7f82cc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.865+0000 7f82d11c2700 1 -- 172.21.15.97:0/1404687345 shutdown_connections 2023-10-08T09:54:59.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.865+0000 7f82d11c2700 1 -- 172.21.15.97:0/1404687345 wait complete. 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.865+0000 7f82d11c2700 1 Processor -- start 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 -- start start 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc068190 0x7f82cc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 0x7f82cc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:59.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1971e0 con 0x7f82cc1067a0 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc197350 con 0x7f82cc068190 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82d11c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc197510 con 0x7f82cc068b30 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55964/0 (socket says 172.21.15.97:55964) 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 -- 172.21.15.97:0/2365588313 learned_addr learned my addr 172.21.15.97:0/2365588313 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82ca59c700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 0x7f82cc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cad9d700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc068190 0x7f82cc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 -- 172.21.15.97:0/2365588313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 msgr2=0x7f82cc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 0x7f82cc19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 -- 172.21.15.97:0/2365588313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc068190 msgr2=0x7f82cc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:54:59.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc068190 0x7f82cc19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 -- 172.21.15.97:0/2365588313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00095f0 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.866+0000 7f82cb59e700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ef30 tx=0x7f82c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.867+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4018070 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.867+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c400f040 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.867+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400cd00 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.867+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc197790 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.867+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc1a3900 con 0x7f82cc1067a0 2023-10-08T09:54:59.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.868+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc108d50 con 0x7f82cc1067a0 2023-10-08T09:54:59.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.869+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f82c4003a00 con 0x7f82cc1067a0 2023-10-08T09:54:59.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.869+0000 7f82b3fff700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 0x7f82b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:54:59.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.869+0000 7f82cad9d700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 0x7f82b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:54:59.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.869+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82c40cd4c0 con 0x7f82cc1067a0 2023-10-08T09:54:59.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.870+0000 7f82cad9d700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 0x7f82b40a3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005950 tx=0x7f82bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:54:59.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:54:59.874+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f82c40968a0 con 0x7f82cc1067a0 2023-10-08T09:54:59.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168609 got 240518168611 for osd.8 2023-10-08T09:54:59.975 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:00.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.081+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f82cc101ee0 con 0x7f82cc1067a0 2023-10-08T09:55:00.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.082+0000 7f82b3fff700 1 -- 172.21.15.97:0/2365588313 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f82c4096030 con 0x7f82cc1067a0 2023-10-08T09:55:00.084 INFO:teuthology.orchestra.run.smithi097.stdout:292057776155 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 msgr2=0x7f82b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 0x7f82b40a3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005950 tx=0x7f82bc00b410 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 msgr2=0x7f82cc196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ef30 tx=0x7f82c400c3a0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 shutdown_connections 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82b40a1480 0x7f82b40a3940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82cc1067a0 0x7f82cc196a30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82cc068190 0x7f82cc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 --2- 172.21.15.97:0/2365588313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82cc068b30 0x7f82cc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 >> 172.21.15.97:0/2365588313 conn(0x7f82cc075790 msgr2=0x7f82cc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:00.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 shutdown_connections 2023-10-08T09:55:00.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.084+0000 7f82d11c2700 1 -- 172.21.15.97:0/2365588313 wait complete. 2023-10-08T09:55:00.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2365588313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:55:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2365588313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:55:00.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2365588313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:55:00.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.405+0000 7f1600fa8700 1 -- 172.21.15.97:0/3324032514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 msgr2=0x7f15fc10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 --2- 172.21.15.97:0/3324032514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc10d650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b0a0 tx=0x7f15f400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 -- 172.21.15.97:0/3324032514 shutdown_connections 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 --2- 172.21.15.97:0/3324032514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc10d650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 --2- 172.21.15.97:0/3324032514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 0x7f15fc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 --2- 172.21.15.97:0/3324032514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15fc1008a0 0x7f15fc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 -- 172.21.15.97:0/3324032514 >> 172.21.15.97:0/3324032514 conn(0x7f15fc077910 msgr2=0x7f15fc077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 -- 172.21.15.97:0/3324032514 shutdown_connections 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.406+0000 7f1600fa8700 1 -- 172.21.15.97:0/3324032514 wait complete. 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 Processor -- start 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 -- start start 2023-10-08T09:55:00.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15fc1008a0 0x7f15fc196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 0x7f15fc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc19bc60 con 0x7f15fc101260 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc19bdd0 con 0x7f15fc1008a0 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f1600fa8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc19bf90 con 0x7f15fc1067f0 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55990/0 (socket says 172.21.15.97:55990) 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 -- 172.21.15.97:0/1466503097 learned_addr learned my addr 172.21.15.97:0/1466503097 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15fad9d700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 0x7f15fc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 -- 172.21.15.97:0/1466503097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 msgr2=0x7f15fc19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 0x7f15fc19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 -- 172.21.15.97:0/1466503097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15fc1008a0 msgr2=0x7f15fc196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15fc1008a0 0x7f15fc196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 -- 172.21.15.97:0/1466503097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009d00 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.407+0000 7f15f9d9b700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e9a0 tx=0x7f15f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.408+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0009790 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.408+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc19c270 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.408+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f000cbd0 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.408+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0018680 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.408+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc1a3ad0 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.409+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc108d30 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.410+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f15f000cd40 con 0x7f15fc101260 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.411+0000 7f15eb7fe700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 0x7f15e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.411+0000 7f15fa59c700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 0x7f15e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.411+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15f0014070 con 0x7f15fc101260 2023-10-08T09:55:00.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.411+0000 7f15fa59c700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 0x7f15e40a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009b60 tx=0x7f15ec009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:00.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.414+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f15f0096800 con 0x7f15fc101260 2023-10-08T09:55:00.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.621+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f15fc101f20 con 0x7f15fc101260 2023-10-08T09:55:00.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.621+0000 7f15eb7fe700 1 -- 172.21.15.97:0/1466503097 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f15f0099e20 con 0x7f15fc101260 2023-10-08T09:55:00.622 INFO:teuthology.orchestra.run.smithi097.stdout:158913790000 2023-10-08T09:55:00.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.623+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 msgr2=0x7f15e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.623+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 0x7f15e40a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009b60 tx=0x7f15ec009620 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.623+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 msgr2=0x7f15fc1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.623+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e9a0 tx=0x7f15f000ecb0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 shutdown_connections 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f15e40a1500 0x7f15e40a39c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15fc1067f0 0x7f15fc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15fc1008a0 0x7f15fc196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 --2- 172.21.15.97:0/1466503097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15fc101260 0x7f15fc1972a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 >> 172.21.15.97:0/1466503097 conn(0x7f15fc077910 msgr2=0x7f15fc0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 shutdown_connections 2023-10-08T09:55:00.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.624+0000 7f1600fa8700 1 -- 172.21.15.97:0/1466503097 wait complete. 2023-10-08T09:55:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.311+0000 7f2d5ff78700 1 -- 172.21.15.97:0/3213979280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d580ff750 msgr2=0x7f2d580ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.311+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/3213979280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d580ff750 0x7f2d580ffbd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009990 tx=0x7f2d48009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- 172.21.15.97:0/3213979280 shutdown_connections 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/3213979280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d58100110 0x7f2d5810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/3213979280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d581056b0 0x7f2d58105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/3213979280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d580ff750 0x7f2d580ffbd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- 172.21.15.97:0/3213979280 >> 172.21.15.97:0/3213979280 conn(0x7f2d58075790 msgr2=0x7f2d58075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- 172.21.15.97:0/3213979280 shutdown_connections 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- 172.21.15.97:0/3213979280 wait complete. 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 Processor -- start 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- start start 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d58100110 0x7f2d58197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 0x7f2d58198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d581a2da0 con 0x7f2d581056b0 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d581a2f10 con 0x7f2d580ff750 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.312+0000 7f2d5ff78700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d581a30d0 con 0x7f2d58100110 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5e515700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 0x7f2d58198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:35944/0 (socket says 172.21.15.97:35944) 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 -- 172.21.15.97:0/2417518861 learned_addr learned my addr 172.21.15.97:0/2417518861 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5d513700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d58100110 0x7f2d58197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 -- 172.21.15.97:0/2417518861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d58100110 msgr2=0x7f2d58197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d58100110 0x7f2d58197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 -- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 msgr2=0x7f2d58198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 0x7f2d58198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 -- 172.21.15.97:0/2417518861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480095f0 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5e515700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 0x7f2d58198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5dd14700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400d980 tx=0x7f2d5400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54004220 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d581a33b0 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d54007640 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54005690 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.313+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d581a38b0 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.314+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58107cd0 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.315+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2d54004750 con 0x7f2d580ff750 2023-10-08T09:55:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.316+0000 7f2d4effd700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 0x7f2d440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.316+0000 7f2d5d513700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 0x7f2d440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.316+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d540cd290 con 0x7f2d580ff750 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.316+0000 7f2d5d513700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 0x7f2d440a3940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d58068950 tx=0x7f2d4800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stdout:55834574912 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.319+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d540965e0 con 0x7f2d580ff750 2023-10-08T09:55:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.533+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d58100df0 con 0x7f2d580ff750 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.534+0000 7f2d4effd700 1 -- 172.21.15.97:0/2417518861 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d54016020 con 0x7f2d580ff750 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 msgr2=0x7f2d440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 0x7f2d440a3940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d58068950 tx=0x7f2d4800b260 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 msgr2=0x7f2d58197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400d980 tx=0x7f2d5400dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 shutdown_connections 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2d440a1480 0x7f2d440a3940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d581056b0 0x7f2d58198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d580ff750 0x7f2d58197a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 --2- 172.21.15.97:0/2417518861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d58100110 0x7f2d58197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 >> 172.21.15.97:0/2417518861 conn(0x7f2d58075790 msgr2=0x7f2d580fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 shutdown_connections 2023-10-08T09:55:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:00.536+0000 7f2d5ff78700 1 -- 172.21.15.97:0/2417518861 wait complete. 2023-10-08T09:55:00.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182453 got 107374182455 for osd.3 2023-10-08T09:55:00.742 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:01.439 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:01.439 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.439 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2417518861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:55:01.440 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1466503097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2417518861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:55:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1466503097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:55:01.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561065 got 188978561068 for osd.6 2023-10-08T09:55:01.495 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: pgmap v294: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2417518861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:55:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1466503097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:55:02.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574910 got 55834574912 for osd.1 2023-10-08T09:55:02.087 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:02.341 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:02.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789998 got 158913790000 for osd.5 2023-10-08T09:55:02.526 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:02.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.747+0000 7f77a19f5700 1 -- 172.21.15.97:0/1504564068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c101600 msgr2=0x7f779c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:02.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.747+0000 7f77a19f5700 1 --2- 172.21.15.97:0/1504564068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c101600 0x7f779c1019e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f778c0099c0 tx=0x7f778c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 -- 172.21.15.97:0/1504564068 shutdown_connections 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 --2- 172.21.15.97:0/1504564068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ffb10 0x7f779c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 --2- 172.21.15.97:0/1504564068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c101600 0x7f779c1019e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 --2- 172.21.15.97:0/1504564068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c0ff150 0x7f779c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 -- 172.21.15.97:0/1504564068 >> 172.21.15.97:0/1504564068 conn(0x7f779c074aa0 msgr2=0x7f779c074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:02.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 -- 172.21.15.97:0/1504564068 shutdown_connections 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.748+0000 7f77a19f5700 1 -- 172.21.15.97:0/1504564068 wait complete. 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 Processor -- start 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 -- start start 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff150 0x7f779c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c101600 0x7f779c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:02.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19dd70 con 0x7f779c0ffb10 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19dee0 con 0x7f779c0ff150 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f77a19f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19e0a0 con 0x7f779c101600 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f779a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f779a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56004/0 (socket says 172.21.15.97:56004) 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f779a7fc700 1 -- 172.21.15.97:0/2260336788 learned_addr learned my addr 172.21.15.97:0/2260336788 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.749+0000 7f779b7fe700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c101600 0x7f779c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:02.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779affd700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff150 0x7f779c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 -- 172.21.15.97:0/2260336788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c101600 msgr2=0x7f779c19d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c101600 0x7f779c19d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 -- 172.21.15.97:0/2260336788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff150 msgr2=0x7f779c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff150 0x7f779c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 -- 172.21.15.97:0/2260336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0095f0 con 0x7f779c0ffb10 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f779a7fc700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f779000c920 tx=0x7f779000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790004a30 con 0x7f779c0ffb10 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c19e380 con 0x7f779c0ffb10 2023-10-08T09:55:02.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77900054e0 con 0x7f779c0ffb10 2023-10-08T09:55:02.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790007c10 con 0x7f779c0ffb10 2023-10-08T09:55:02.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.750+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c1a5be0 con 0x7f779c0ffb10 2023-10-08T09:55:02.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.751+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c10b040 con 0x7f779c0ffb10 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.753+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7790005000 con 0x7f779c0ffb10 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.753+0000 7f77a09f3700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 0x7f77840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.753+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77900ccf00 con 0x7f779c0ffb10 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.753+0000 7f779affd700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 0x7f77840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.754+0000 7f779affd700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 0x7f77840a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f778c009f50 tx=0x7f778c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.757+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7790096250 con 0x7f779c0ffb10 2023-10-08T09:55:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.956+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f779c19a180 con 0x7f779c0ffb10 2023-10-08T09:55:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.956+0000 7f77a09f3700 1 -- 172.21.15.97:0/2260336788 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f779001d090 con 0x7f779c0ffb10 2023-10-08T09:55:02.957 INFO:teuthology.orchestra.run.smithi097.stdout:270582939680 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 msgr2=0x7f77840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 0x7f77840a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f778c009f50 tx=0x7f778c005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 msgr2=0x7f779c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f779000c920 tx=0x7f779000cc30 comp rx=0 tx=0).stop 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 shutdown_connections 2023-10-08T09:55:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f77840a1480 0x7f77840a3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f779c101600 0x7f779c19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff150 0x7f779c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 --2- 172.21.15.97:0/2260336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f779c0ffb10 0x7f779c1993b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 >> 172.21.15.97:0/2260336788 conn(0x7f779c074aa0 msgr2=0x7f779c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 shutdown_connections 2023-10-08T09:55:02.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:02.958+0000 7f77a19f5700 1 -- 172.21.15.97:0/2260336788 wait complete. 2023-10-08T09:55:03.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:03 smithi097 ceph-mon[99491]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2260336788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:55:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:03 smithi178 ceph-mon[103156]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:03.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:03 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2260336788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:55:03.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776153 got 292057776155 for osd.10 2023-10-08T09:55:03.481 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:03 smithi186 ceph-mon[104393]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 323 B/s, 0 objects/s recovering 2023-10-08T09:55:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:03 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2260336788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:55:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 -- 172.21.15.97:0/2600381840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000100c60 msgr2=0x7f900010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 --2- 172.21.15.97:0/2600381840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000100c60 0x7f900010c5c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b0a0 tx=0x7f8ff800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 -- 172.21.15.97:0/2600381840 shutdown_connections 2023-10-08T09:55:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 --2- 172.21.15.97:0/2600381840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000100c60 0x7f900010c5c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 --2- 172.21.15.97:0/2600381840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000104720 0x7f9000104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 --2- 172.21.15.97:0/2600381840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90001002a0 0x7f9000100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 -- 172.21.15.97:0/2600381840 >> 172.21.15.97:0/2600381840 conn(0x7f9000074a10 msgr2=0x7f9000074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 -- 172.21.15.97:0/2600381840 shutdown_connections 2023-10-08T09:55:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.794+0000 7f9006c81700 1 -- 172.21.15.97:0/2600381840 wait complete. 2023-10-08T09:55:03.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 Processor -- start 2023-10-08T09:55:03.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 -- start start 2023-10-08T09:55:03.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001002a0 0x7f9000197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:03.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 0x7f9000198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001988d0 con 0x7f9000104720 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001a30e0 con 0x7f9000100c60 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9006c81700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001a3220 con 0x7f90001002a0 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56018/0 (socket says 172.21.15.97:56018) 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 -- 172.21.15.97:0/2972552833 learned_addr learned my addr 172.21.15.97:0/2972552833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f8ffffff700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 0x7f9000198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f9004a1d700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001002a0 0x7f9000197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 -- 172.21.15.97:0/2972552833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001002a0 msgr2=0x7f9000197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001002a0 0x7f9000197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 -- 172.21.15.97:0/2972552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 msgr2=0x7f9000198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 0x7f9000198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 -- 172.21.15.97:0/2972552833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8009d00 con 0x7f9000104720 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f8ffffff700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 0x7f9000198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.795+0000 7f900521e700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b830 tx=0x7f8ff80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.796+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800e040 con 0x7f9000104720 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.796+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff8004e60 con 0x7f9000104720 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.796+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90001a3450 con 0x7f9000104720 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.796+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8005420 con 0x7f9000104720 2023-10-08T09:55:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.796+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001a3870 con 0x7f9000104720 2023-10-08T09:55:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.797+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000107c60 con 0x7f9000104720 2023-10-08T09:55:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.798+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ff801bb30 con 0x7f9000104720 2023-10-08T09:55:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.799+0000 7f8ffdffb700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 0x7f8ff00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.799+0000 7f9004a1d700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 0x7f8ff00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.799+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8ff80ce9f0 con 0x7f9000104720 2023-10-08T09:55:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.799+0000 7f9004a1d700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 0x7f8ff00a3940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005fd0 tx=0x7f8ff4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.803+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ff8097dd0 con 0x7f9000104720 2023-10-08T09:55:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.999+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f900010f700 con 0x7f9000104720 2023-10-08T09:55:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:03.999+0000 7f8ffdffb700 1 -- 172.21.15.97:0/2972552833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ff809b3f0 con 0x7f9000104720 2023-10-08T09:55:04.001 INFO:teuthology.orchestra.run.smithi097.stdout:137438953525 2023-10-08T09:55:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.001+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 msgr2=0x7f8ff00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.001+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 0x7f8ff00a3940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005fd0 tx=0x7f8ff4005db0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.001+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 msgr2=0x7f90001a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.001+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b830 tx=0x7f8ff80078a0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 shutdown_connections 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ff00a1480 0x7f8ff00a3940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000104720 0x7f90001a2ba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001002a0 0x7f9000197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 --2- 172.21.15.97:0/2972552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9000100c60 0x7f9000198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 >> 172.21.15.97:0/2972552833 conn(0x7f9000074a10 msgr2=0x7f90000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 shutdown_connections 2023-10-08T09:55:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.002+0000 7f9006c81700 1 -- 172.21.15.97:0/2972552833 wait complete. 2023-10-08T09:55:04.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939678 got 270582939680 for osd.9 2023-10-08T09:55:04.249 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:04.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:04 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2972552833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:55:04.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:04 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2972552833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:55:04.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:04 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2972552833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:55:04.713 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953522 got 137438953525 for osd.4 2023-10-08T09:55:04.714 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- 172.21.15.97:0/429901956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 msgr2=0x7f9ea810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 --2- 172.21.15.97:0/429901956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea810d650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000b0d0 tx=0x7f9ea000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- 172.21.15.97:0/429901956 shutdown_connections 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 --2- 172.21.15.97:0/429901956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea810d650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 --2- 172.21.15.97:0/429901956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 0x7f9ea8105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 --2- 172.21.15.97:0/429901956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea8068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- 172.21.15.97:0/429901956 >> 172.21.15.97:0/429901956 conn(0x7f9ea80766c0 msgr2=0x7f9ea8076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- 172.21.15.97:0/429901956 shutdown_connections 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- 172.21.15.97:0/429901956 wait complete. 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 Processor -- start 2023-10-08T09:55:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.800+0000 7f9eaf900700 1 -- start start 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea80ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 0x7f9ea8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a5070 con 0x7f9ea8069410 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a51e0 con 0x7f9ea8068a50 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81a53a0 con 0x7f9ea8105670 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56030/0 (socket says 172.21.15.97:56030) 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 -- 172.21.15.97:0/328105030 learned_addr learned my addr 172.21.15.97:0/328105030 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9ead69c700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea80ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eade9d700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 0x7f9ea8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 -- 172.21.15.97:0/328105030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 msgr2=0x7f9ea8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 0x7f9ea8100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 -- 172.21.15.97:0/328105030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 msgr2=0x7f9ea80ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea80ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 -- 172.21.15.97:0/328105030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0009d00 con 0x7f9ea8069410 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9ead69c700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea80ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eace9b700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800d950 tx=0x7f9e9800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98009b40 con 0x7f9ea8069410 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e98009ca0 con 0x7f9ea8069410 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81a5680 con 0x7f9ea8069410 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800f620 con 0x7f9ea8069410 2023-10-08T09:55:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.801+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81a5bd0 con 0x7f9ea8069410 2023-10-08T09:55:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.802+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea8108d30 con 0x7f9ea8069410 2023-10-08T09:55:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.803+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9e98010460 con 0x7f9ea8069410 2023-10-08T09:55:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.804+0000 7f9e9e7fc700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 0x7f9e940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.804+0000 7f9ead69c700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 0x7f9e940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:04.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.804+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9e980cd310 con 0x7f9ea8069410 2023-10-08T09:55:04.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.804+0000 7f9ead69c700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 0x7f9e940a39c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005fd0 tx=0x7f9ea400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:04.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:04.808+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e98096660 con 0x7f9ea8069410 2023-10-08T09:55:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.017+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9ea804e4f0 con 0x7f9ea8069410 2023-10-08T09:55:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.018+0000 7f9e9e7fc700 1 -- 172.21.15.97:0/328105030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e98099c80 con 0x7f9ea8069410 2023-10-08T09:55:05.019 INFO:teuthology.orchestra.run.smithi097.stdout:77309411389 2023-10-08T09:55:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 msgr2=0x7f9e940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 0x7f9e940a39c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005fd0 tx=0x7f9ea400a5e0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 msgr2=0x7f9ea80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800d950 tx=0x7f9e9800dd10 comp rx=0 tx=0).stop 2023-10-08T09:55:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 shutdown_connections 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9e940a1500 0x7f9e940a39c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ea8105670 0x7f9ea8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea8068a50 0x7f9ea80ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 --2- 172.21.15.97:0/328105030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8069410 0x7f9ea80ffcf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 >> 172.21.15.97:0/328105030 conn(0x7f9ea80766c0 msgr2=0x7f9ea80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 shutdown_connections 2023-10-08T09:55:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.020+0000 7f9eaf900700 1 -- 172.21.15.97:0/328105030 wait complete. 2023-10-08T09:55:05.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 -- 172.21.15.97:0/3335284209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541008a0 msgr2=0x7f8654100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 --2- 172.21.15.97:0/3335284209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541008a0 0x7f8654100d20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8644009990 tx=0x7f8644009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 -- 172.21.15.97:0/3335284209 shutdown_connections 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 --2- 172.21.15.97:0/3335284209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 --2- 172.21.15.97:0/3335284209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541057c0 0x7f8654105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 --2- 172.21.15.97:0/3335284209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541008a0 0x7f8654100d20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 -- 172.21.15.97:0/3335284209 >> 172.21.15.97:0/3335284209 conn(0x7f86540779e0 msgr2=0x7f8654077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.154+0000 7f865b8b1700 1 -- 172.21.15.97:0/3335284209 shutdown_connections 2023-10-08T09:55:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.155+0000 7f865b8b1700 1 -- 172.21.15.97:0/3335284209 wait complete. 2023-10-08T09:55:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.155+0000 7f865b8b1700 1 Processor -- start 2023-10-08T09:55:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.155+0000 7f865b8b1700 1 -- start start 2023-10-08T09:55:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 0x7f865419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654197100 con 0x7f86541057c0 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654197270 con 0x7f8654101260 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654197430 con 0x7f86541008a0 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56046/0 (socket says 172.21.15.97:56046) 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 -- 172.21.15.97:0/710882062 learned_addr learned my addr 172.21.15.97:0/710882062 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8658e4c700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865964d700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 0x7f865419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 -- 172.21.15.97:0/710882062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 msgr2=0x7f865419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 0x7f865419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 -- 172.21.15.97:0/710882062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 msgr2=0x7f865419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 -- 172.21.15.97:0/710882062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440095f0 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8658e4c700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865964d700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 0x7f865419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f8659e4e700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f864c00efc0 tx=0x7f864c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c018070 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864c00f040 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00cd80 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86541976b0 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.156+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86541a3950 con 0x7f86541057c0 2023-10-08T09:55:05.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.157+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654108da0 con 0x7f86541057c0 2023-10-08T09:55:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.159+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f864c003a00 con 0x7f86541057c0 2023-10-08T09:55:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.159+0000 7f864a7fc700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 0x7f86400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.159+0000 7f865964d700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 0x7f86400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.159+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f864c0cd480 con 0x7f86541057c0 2023-10-08T09:55:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.160+0000 7f865964d700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 0x7f86400a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8650005e90 tx=0x7f8650005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.163+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f864c096860 con 0x7f86541057c0 2023-10-08T09:55:05.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.374+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f8654197f60 con 0x7f86541057c0 2023-10-08T09:55:05.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.374+0000 7f864a7fc700 1 -- 172.21.15.97:0/710882062 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f864c099e80 con 0x7f86541057c0 2023-10-08T09:55:05.375 INFO:teuthology.orchestra.run.smithi097.stdout:326417514521 2023-10-08T09:55:05.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 msgr2=0x7f86400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 0x7f86400a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8650005e90 tx=0x7f8650005d40 comp rx=0 tx=0).stop 2023-10-08T09:55:05.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 msgr2=0x7f8654196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f864c00efc0 tx=0x7f864c00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 shutdown_connections 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86400a1480 0x7f86400a3940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86541057c0 0x7f8654196aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86541008a0 0x7f865419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 --2- 172.21.15.97:0/710882062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8654101260 0x7f865419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 >> 172.21.15.97:0/710882062 conn(0x7f86540779e0 msgr2=0x7f86540fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 shutdown_connections 2023-10-08T09:55:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:05.376+0000 7f865b8b1700 1 -- 172.21.15.97:0/710882062 wait complete. 2023-10-08T09:55:05.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:05 smithi097 ceph-mon[99491]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 3 B/s, 0 objects/s recovering 2023-10-08T09:55:05.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:05 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/328105030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:55:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:05 smithi178 ceph-mon[103156]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 3 B/s, 0 objects/s recovering 2023-10-08T09:55:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:05 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/328105030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:55:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:05 smithi186 ceph-mon[104393]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 3 B/s, 0 objects/s recovering 2023-10-08T09:55:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:05 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/328105030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:55:05.715 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411389 for osd.2 2023-10-08T09:55:05.715 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:05.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514518 got 326417514521 for osd.11 2023-10-08T09:55:05.969 DEBUG:teuthology.parallel:result is None 2023-10-08T09:55:05.969 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-08T09:55:05.970 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T09:55:06.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:06 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/710882062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:55:06.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:06 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/710882062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:55:06.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:06 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/710882062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:55:06.535 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:07.310 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:07 smithi097 ceph-mon[99491]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:07 smithi178 ceph-mon[103156]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:07.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:07 smithi186 ceph-mon[104393]: pgmap v297: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 -- 172.21.15.97:0/2297941811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a80068b30 msgr2=0x7f2a8010d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/2297941811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a80068b30 0x7f2a8010d670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b0a0 tx=0x7f2a7800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 -- 172.21.15.97:0/2297941811 shutdown_connections 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/2297941811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a80068b30 0x7f2a8010d670 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/2297941811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a801067a0 0x7f2a80106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/2297941811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068190 0x7f2a800685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 -- 172.21.15.97:0/2297941811 >> 172.21.15.97:0/2297941811 conn(0x7f2a80075790 msgr2=0x7f2a80075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 -- 172.21.15.97:0/2297941811 shutdown_connections 2023-10-08T09:55:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 -- 172.21.15.97:0/2297941811 wait complete. 2023-10-08T09:55:07.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.744+0000 7f2a85e8b700 1 Processor -- start 2023-10-08T09:55:07.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 -- start start 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a80068190 0x7f2a800ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 0x7f2a800ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801a50e0 con 0x7f2a801067a0 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801a5250 con 0x7f2a80068190 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a85e8b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801a5410 con 0x7f2a80068b30 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a7ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a7ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56056/0 (socket says 172.21.15.97:56056) 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a7ffff700 1 -- 172.21.15.97:0/470894169 learned_addr learned my addr 172.21.15.97:0/470894169 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a7f7fe700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a80068190 0x7f2a800ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.745+0000 7f2a7effd700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 0x7f2a800ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 -- 172.21.15.97:0/470894169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 msgr2=0x7f2a800ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 0x7f2a800ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 -- 172.21.15.97:0/470894169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a80068190 msgr2=0x7f2a800ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a80068190 0x7f2a800ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 -- 172.21.15.97:0/470894169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78009d00 con 0x7f2a801067a0 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7effd700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 0x7f2a800ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7ffff700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b070 tx=0x7f2a78007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800e040 con 0x7f2a801067a0 2023-10-08T09:55:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a78009440 con 0x7f2a801067a0 2023-10-08T09:55:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a801a5690 con 0x7f2a801067a0 2023-10-08T09:55:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7801b450 con 0x7f2a801067a0 2023-10-08T09:55:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.746+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a801a5b50 con 0x7f2a801067a0 2023-10-08T09:55:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.747+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a80108d50 con 0x7f2a801067a0 2023-10-08T09:55:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.748+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2a78004cf0 con 0x7f2a801067a0 2023-10-08T09:55:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.749+0000 7f2a7cff9700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 0x7f2a680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.749+0000 7f2a7f7fe700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 0x7f2a680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.749+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a780ce4b0 con 0x7f2a801067a0 2023-10-08T09:55:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.749+0000 7f2a7f7fe700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 0x7f2a680a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007b60 tx=0x7f2a700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.753+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2a78097890 con 0x7f2a801067a0 2023-10-08T09:55:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.965+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a80100d00 con 0x7f2a680a1500 2023-10-08T09:55:07.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.978+0000 7f2a7cff9700 1 -- 172.21.15.97:0/470894169 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427454 (secure 0 0 0) 0x7f2a80100d00 con 0x7f2a680a1500 2023-10-08T09:55:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 msgr2=0x7f2a680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 0x7f2a680a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007b60 tx=0x7f2a700073d0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 msgr2=0x7f2a801002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b070 tx=0x7f2a78007870 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 shutdown_connections 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2a680a1500 0x7f2a680a39c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a801067a0 0x7f2a801002a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a80068190 0x7f2a800ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 --2- 172.21.15.97:0/470894169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a80068b30 0x7f2a800ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 >> 172.21.15.97:0/470894169 conn(0x7f2a80075790 msgr2=0x7f2a800fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:07.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 shutdown_connections 2023-10-08T09:55:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:07.982+0000 7f2a85e8b700 1 -- 172.21.15.97:0/470894169 wait complete. 2023-10-08T09:55:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:08.398 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-10-08T09:55:07.895812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":388,"num_read_kb":253,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337572,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604252,"statfs":{"total":1151940427776,"available":1151594754048,"internally_reserved":0,"allocated":13627392,"data_stored":7498200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003625"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242921+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:46.242921+0000","last_peered":"2023-10-08T09:54:46.242921+0000","last_clean":"2023-10-08T09:54:46.242921+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:46.242921+0000","last_undegraded":"2023-10-08T09:54:46.242921+0000","last_fullsized":"2023-10-08T09:54:46.242921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239846+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:46.239846+0000","last_peered":"2023-10-08T09:54:46.239846+0000","last_clean":"2023-10-08T09:54:46.239846+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:46.239846+0000","last_undegraded":"2023-10-08T09:54:46.239846+0000","last_fullsized":"2023-10-08T09:54:46.239846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240381+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:46.240381+0000","last_peered":"2023-10-08T09:54:46.240381+0000","last_clean":"2023-10-08T09:54:46.240381+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:46.240381+0000","last_undegraded":"2023-10-08T09:54:46.240381+0000","last_fullsized":"2023-10-08T09:54:46.240381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235313+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:46.235313+0000","last_peered":"2023-10-08T09:54:46.235313+0000","last_clean":"2023-10-08T09:54:46.235313+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:46.235313+0000","last_undegraded":"2023-10-08T09:54:46.235313+0000","last_fullsized":"2023-10-08T09:54:46.235313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237260+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:54:46.237260+0000","last_peered":"2023-10-08T09:54:46.237260+0000","last_clean":"2023-10-08T09:54:46.237260+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:54:46.237260+0000","last_undegraded":"2023-10-08T09:54:46.237260+0000","last_fullsized":"2023-10-08T09:54:46.237260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239591+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:46.239591+0000","last_peered":"2023-10-08T09:54:46.239591+0000","last_clean":"2023-10-08T09:54:46.239591+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:46.239591+0000","last_undegraded":"2023-10-08T09:54:46.239591+0000","last_fullsized":"2023-10-08T09:54:46.239591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239548+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:46.239548+0000","last_peered":"2023-10-08T09:54:46.239548+0000","last_clean":"2023-10-08T09:54:46.239548+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:46.239548+0000","last_undegraded":"2023-10-08T09:54:46.239548+0000","last_fullsized":"2023-10-08T09:54:46.239548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235925+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:54:46.235925+0000","last_peered":"2023-10-08T09:54:46.235925+0000","last_clean":"2023-10-08T09:54:46.235925+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:54:46.235925+0000","last_undegraded":"2023-10-08T09:54:46.235925+0000","last_fullsized":"2023-10-08T09:54:46.235925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242828+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:46.242828+0000","last_peered":"2023-10-08T09:54:46.242828+0000","last_clean":"2023-10-08T09:54:46.242828+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:46.242828+0000","last_undegraded":"2023-10-08T09:54:46.242828+0000","last_fullsized":"2023-10-08T09:54:46.242828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238442+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:46.238442+0000","last_peered":"2023-10-08T09:54:46.238442+0000","last_clean":"2023-10-08T09:54:46.238442+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:46.238442+0000","last_undegraded":"2023-10-08T09:54:46.238442+0000","last_fullsized":"2023-10-08T09:54:46.238442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239781+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:46.239781+0000","last_peered":"2023-10-08T09:54:46.239781+0000","last_clean":"2023-10-08T09:54:46.239781+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:46.239781+0000","last_undegraded":"2023-10-08T09:54:46.239781+0000","last_fullsized":"2023-10-08T09:54:46.239781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372615+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:54:47.372615+0000","last_peered":"2023-10-08T09:54:47.372615+0000","last_clean":"2023-10-08T09:54:47.372615+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:54:47.372615+0000","last_undegraded":"2023-10-08T09:54:47.372615+0000","last_fullsized":"2023-10-08T09:54:47.372615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238188+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:46.238188+0000","last_peered":"2023-10-08T09:54:46.238188+0000","last_clean":"2023-10-08T09:54:46.238188+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:46.238188+0000","last_undegraded":"2023-10-08T09:54:46.238188+0000","last_fullsized":"2023-10-08T09:54:46.238188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235877+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:46.235877+0000","last_peered":"2023-10-08T09:54:46.235877+0000","last_clean":"2023-10-08T09:54:46.235877+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:46.235877+0000","last_undegraded":"2023-10-08T09:54:46.235877+0000","last_fullsized":"2023-10-08T09:54:46.235877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235918+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:46.235918+0000","last_peered":"2023-10-08T09:54:46.235918+0000","last_clean":"2023-10-08T09:54:46.235918+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:46.235918+0000","last_undegraded":"2023-10-08T09:54:46.235918+0000","last_fullsized":"2023-10-08T09:54:46.235918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241439+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:46.241439+0000","last_peered":"2023-10-08T09:54:46.241439+0000","last_clean":"2023-10-08T09:54:46.241439+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:46.241439+0000","last_undegraded":"2023-10-08T09:54:46.241439+0000","last_fullsized":"2023-10-08T09:54:46.241439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236163+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:54:46.236163+0000","last_peered":"2023-10-08T09:54:46.236163+0000","last_clean":"2023-10-08T09:54:46.236163+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:54:46.236163+0000","last_undegraded":"2023-10-08T09:54:46.236163+0000","last_fullsized":"2023-10-08T09:54:46.236163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239642+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:46.239642+0000","last_peered":"2023-10-08T09:54:46.239642+0000","last_clean":"2023-10-08T09:54:46.239642+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:46.239642+0000","last_undegraded":"2023-10-08T09:54:46.239642+0000","last_fullsized":"2023-10-08T09:54:46.239642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:03.249827+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:03.249827+0000","last_peered":"2023-10-08T09:55:03.249827+0000","last_clean":"2023-10-08T09:55:03.249827+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:03.249827+0000","last_undegraded":"2023-10-08T09:55:03.249827+0000","last_fullsized":"2023-10-08T09:55:03.249827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239716+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:46.239716+0000","last_peered":"2023-10-08T09:54:46.239716+0000","last_clean":"2023-10-08T09:54:46.239716+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:46.239716+0000","last_undegraded":"2023-10-08T09:54:46.239716+0000","last_fullsized":"2023-10-08T09:54:46.239716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242686+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:46.242686+0000","last_peered":"2023-10-08T09:54:46.242686+0000","last_clean":"2023-10-08T09:54:46.242686+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:46.242686+0000","last_undegraded":"2023-10-08T09:54:46.242686+0000","last_fullsized":"2023-10-08T09:54:46.242686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235223+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:46.235223+0000","last_peered":"2023-10-08T09:54:46.235223+0000","last_clean":"2023-10-08T09:54:46.235223+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:46.235223+0000","last_undegraded":"2023-10-08T09:54:46.235223+0000","last_fullsized":"2023-10-08T09:54:46.235223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239764+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:46.239764+0000","last_peered":"2023-10-08T09:54:46.239764+0000","last_clean":"2023-10-08T09:54:46.239764+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:46.239764+0000","last_undegraded":"2023-10-08T09:54:46.239764+0000","last_fullsized":"2023-10-08T09:54:46.239764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237422+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:46.237422+0000","last_peered":"2023-10-08T09:54:46.237422+0000","last_clean":"2023-10-08T09:54:46.237422+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:46.237422+0000","last_undegraded":"2023-10-08T09:54:46.237422+0000","last_fullsized":"2023-10-08T09:54:46.237422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235537+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:46.235537+0000","last_peered":"2023-10-08T09:54:46.235537+0000","last_clean":"2023-10-08T09:54:46.235537+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:46.235537+0000","last_undegraded":"2023-10-08T09:54:46.235537+0000","last_fullsized":"2023-10-08T09:54:46.235537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:04.250636+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:04.250636+0000","last_peered":"2023-10-08T09:55:04.250636+0000","last_clean":"2023-10-08T09:55:04.250636+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:04.250636+0000","last_undegraded":"2023-10-08T09:55:04.250636+0000","last_fullsized":"2023-10-08T09:55:04.250636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236359+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:54:46.236359+0000","last_peered":"2023-10-08T09:54:46.236359+0000","last_clean":"2023-10-08T09:54:46.236359+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:54:46.236359+0000","last_undegraded":"2023-10-08T09:54:46.236359+0000","last_fullsized":"2023-10-08T09:54:46.236359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237184+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:46.237184+0000","last_peered":"2023-10-08T09:54:46.237184+0000","last_clean":"2023-10-08T09:54:46.237184+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:46.237184+0000","last_undegraded":"2023-10-08T09:54:46.237184+0000","last_fullsized":"2023-10-08T09:54:46.237184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241365+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:46.241365+0000","last_peered":"2023-10-08T09:54:46.241365+0000","last_clean":"2023-10-08T09:54:46.241365+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:46.241365+0000","last_undegraded":"2023-10-08T09:54:46.241365+0000","last_fullsized":"2023-10-08T09:54:46.241365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236355+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:54:46.236355+0000","last_peered":"2023-10-08T09:54:46.236355+0000","last_clean":"2023-10-08T09:54:46.236355+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:54:46.236355+0000","last_undegraded":"2023-10-08T09:54:46.236355+0000","last_fullsized":"2023-10-08T09:54:46.236355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237737+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:46.237737+0000","last_peered":"2023-10-08T09:54:46.237737+0000","last_clean":"2023-10-08T09:54:46.237737+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:46.237737+0000","last_undegraded":"2023-10-08T09:54:46.237737+0000","last_fullsized":"2023-10-08T09:54:46.237737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'12","reported_seq":137,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:05.273176+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:05.273176+0000","last_peered":"2023-10-08T09:55:05.273176+0000","last_clean":"2023-10-08T09:55:05.273176+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:05.273176+0000","last_undegraded":"2023-10-08T09:55:05.273176+0000","last_fullsized":"2023-10-08T09:55:05.273176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":44,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240097+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:46.240097+0000","last_peered":"2023-10-08T09:54:46.240097+0000","last_clean":"2023-10-08T09:54:46.240097+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:46.240097+0000","last_undegraded":"2023-10-08T09:54:46.240097+0000","last_fullsized":"2023-10-08T09:54:46.240097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242488+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:46.242488+0000","last_peered":"2023-10-08T09:54:46.242488+0000","last_clean":"2023-10-08T09:54:46.242488+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:46.242488+0000","last_undegraded":"2023-10-08T09:54:46.242488+0000","last_fullsized":"2023-10-08T09:54:46.242488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238020+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:46.238020+0000","last_peered":"2023-10-08T09:54:46.238020+0000","last_clean":"2023-10-08T09:54:46.238020+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:46.238020+0000","last_undegraded":"2023-10-08T09:54:46.238020+0000","last_fullsized":"2023-10-08T09:54:46.238020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:03.249309+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:03.249309+0000","last_peered":"2023-10-08T09:55:03.249309+0000","last_clean":"2023-10-08T09:55:03.249309+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:03.249309+0000","last_undegraded":"2023-10-08T09:55:03.249309+0000","last_fullsized":"2023-10-08T09:55:03.249309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.267960+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.267960+0000","last_peered":"2023-10-08T09:54:47.267960+0000","last_clean":"2023-10-08T09:54:47.267960+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.267960+0000","last_undegraded":"2023-10-08T09:54:47.267960+0000","last_fullsized":"2023-10-08T09:54:47.267960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242804+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:46.242804+0000","last_peered":"2023-10-08T09:54:46.242804+0000","last_clean":"2023-10-08T09:54:46.242804+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:46.242804+0000","last_undegraded":"2023-10-08T09:54:46.242804+0000","last_fullsized":"2023-10-08T09:54:46.242804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237811+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:46.237811+0000","last_peered":"2023-10-08T09:54:46.237811+0000","last_clean":"2023-10-08T09:54:46.237811+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:46.237811+0000","last_undegraded":"2023-10-08T09:54:46.237811+0000","last_fullsized":"2023-10-08T09:54:46.237811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:04.249955+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:04.249955+0000","last_peered":"2023-10-08T09:55:04.249955+0000","last_clean":"2023-10-08T09:55:04.249955+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:04.249955+0000","last_undegraded":"2023-10-08T09:55:04.249955+0000","last_fullsized":"2023-10-08T09:55:04.249955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235354+0000","last_change":"2023-10-08T09:53:56.887772+0000","last_active":"2023-10-08T09:54:46.235354+0000","last_peered":"2023-10-08T09:54:46.235354+0000","last_clean":"2023-10-08T09:54:46.235354+0000","last_became_active":"2023-10-08T09:53:56.887155+0000","last_became_peered":"2023-10-08T09:53:56.887155+0000","last_unstale":"2023-10-08T09:54:46.235354+0000","last_undegraded":"2023-10-08T09:54:46.235354+0000","last_fullsized":"2023-10-08T09:54:46.235354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:45.184494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242732+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:46.242732+0000","last_peered":"2023-10-08T09:54:46.242732+0000","last_clean":"2023-10-08T09:54:46.242732+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:46.242732+0000","last_undegraded":"2023-10-08T09:54:46.242732+0000","last_fullsized":"2023-10-08T09:54:46.242732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264240+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264240+0000","last_peered":"2023-10-08T09:54:47.264240+0000","last_clean":"2023-10-08T09:54:47.264240+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264240+0000","last_undegraded":"2023-10-08T09:54:47.264240+0000","last_fullsized":"2023-10-08T09:54:47.264240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235224+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:46.235224+0000","last_peered":"2023-10-08T09:54:46.235224+0000","last_clean":"2023-10-08T09:54:46.235224+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:46.235224+0000","last_undegraded":"2023-10-08T09:54:46.235224+0000","last_fullsized":"2023-10-08T09:54:46.235224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238500+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:46.238500+0000","last_peered":"2023-10-08T09:54:46.238500+0000","last_clean":"2023-10-08T09:54:46.238500+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:46.238500+0000","last_undegraded":"2023-10-08T09:54:46.238500+0000","last_fullsized":"2023-10-08T09:54:46.238500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235824+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:46.235824+0000","last_peered":"2023-10-08T09:54:46.235824+0000","last_clean":"2023-10-08T09:54:46.235824+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:46.235824+0000","last_undegraded":"2023-10-08T09:54:46.235824+0000","last_fullsized":"2023-10-08T09:54:46.235824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239283+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:46.239283+0000","last_peered":"2023-10-08T09:54:46.239283+0000","last_clean":"2023-10-08T09:54:46.239283+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:46.239283+0000","last_undegraded":"2023-10-08T09:54:46.239283+0000","last_fullsized":"2023-10-08T09:54:46.239283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241730+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:46.241730+0000","last_peered":"2023-10-08T09:54:46.241730+0000","last_clean":"2023-10-08T09:54:46.241730+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:46.241730+0000","last_undegraded":"2023-10-08T09:54:46.241730+0000","last_fullsized":"2023-10-08T09:54:46.241730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235841+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:54:46.235841+0000","last_peered":"2023-10-08T09:54:46.235841+0000","last_clean":"2023-10-08T09:54:46.235841+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:54:46.235841+0000","last_undegraded":"2023-10-08T09:54:46.235841+0000","last_fullsized":"2023-10-08T09:54:46.235841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235382+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:46.235382+0000","last_peered":"2023-10-08T09:54:46.235382+0000","last_clean":"2023-10-08T09:54:46.235382+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:46.235382+0000","last_undegraded":"2023-10-08T09:54:46.235382+0000","last_fullsized":"2023-10-08T09:54:46.235382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242474+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:46.242474+0000","last_peered":"2023-10-08T09:54:46.242474+0000","last_clean":"2023-10-08T09:54:46.242474+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:46.242474+0000","last_undegraded":"2023-10-08T09:54:46.242474+0000","last_fullsized":"2023-10-08T09:54:46.242474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239156+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:46.239156+0000","last_peered":"2023-10-08T09:54:46.239156+0000","last_clean":"2023-10-08T09:54:46.239156+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:46.239156+0000","last_undegraded":"2023-10-08T09:54:46.239156+0000","last_fullsized":"2023-10-08T09:54:46.239156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235543+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:46.235543+0000","last_peered":"2023-10-08T09:54:46.235543+0000","last_clean":"2023-10-08T09:54:46.235543+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:46.235543+0000","last_undegraded":"2023-10-08T09:54:46.235543+0000","last_fullsized":"2023-10-08T09:54:46.235543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235875+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:54:46.235875+0000","last_peered":"2023-10-08T09:54:46.235875+0000","last_clean":"2023-10-08T09:54:46.235875+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:54:46.235875+0000","last_undegraded":"2023-10-08T09:54:46.235875+0000","last_fullsized":"2023-10-08T09:54:46.235875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240249+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:46.240249+0000","last_peered":"2023-10-08T09:54:46.240249+0000","last_clean":"2023-10-08T09:54:46.240249+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:46.240249+0000","last_undegraded":"2023-10-08T09:54:46.240249+0000","last_fullsized":"2023-10-08T09:54:46.240249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239715+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:46.239715+0000","last_peered":"2023-10-08T09:54:46.239715+0000","last_clean":"2023-10-08T09:54:46.239715+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:46.239715+0000","last_undegraded":"2023-10-08T09:54:46.239715+0000","last_fullsized":"2023-10-08T09:54:46.239715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242966+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:46.242966+0000","last_peered":"2023-10-08T09:54:46.242966+0000","last_clean":"2023-10-08T09:54:46.242966+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:46.242966+0000","last_undegraded":"2023-10-08T09:54:46.242966+0000","last_fullsized":"2023-10-08T09:54:46.242966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239466+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:46.239466+0000","last_peered":"2023-10-08T09:54:46.239466+0000","last_clean":"2023-10-08T09:54:46.239466+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:46.239466+0000","last_undegraded":"2023-10-08T09:54:46.239466+0000","last_fullsized":"2023-10-08T09:54:46.239466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239080+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:46.239080+0000","last_peered":"2023-10-08T09:54:46.239080+0000","last_clean":"2023-10-08T09:54:46.239080+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:46.239080+0000","last_undegraded":"2023-10-08T09:54:46.239080+0000","last_fullsized":"2023-10-08T09:54:46.239080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239994+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:46.239994+0000","last_peered":"2023-10-08T09:54:46.239994+0000","last_clean":"2023-10-08T09:54:46.239994+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:46.239994+0000","last_undegraded":"2023-10-08T09:54:46.239994+0000","last_fullsized":"2023-10-08T09:54:46.239994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243338+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:46.243338+0000","last_peered":"2023-10-08T09:54:46.243338+0000","last_clean":"2023-10-08T09:54:46.243338+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:46.243338+0000","last_undegraded":"2023-10-08T09:54:46.243338+0000","last_fullsized":"2023-10-08T09:54:46.243338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239967+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:46.239967+0000","last_peered":"2023-10-08T09:54:46.239967+0000","last_clean":"2023-10-08T09:54:46.239967+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:46.239967+0000","last_undegraded":"2023-10-08T09:54:46.239967+0000","last_fullsized":"2023-10-08T09:54:46.239967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235937+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:46.235937+0000","last_peered":"2023-10-08T09:54:46.235937+0000","last_clean":"2023-10-08T09:54:46.235937+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:46.235937+0000","last_undegraded":"2023-10-08T09:54:46.235937+0000","last_fullsized":"2023-10-08T09:54:46.235937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235296+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:46.235296+0000","last_peered":"2023-10-08T09:54:46.235296+0000","last_clean":"2023-10-08T09:54:46.235296+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:46.235296+0000","last_undegraded":"2023-10-08T09:54:46.235296+0000","last_fullsized":"2023-10-08T09:54:46.235296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236036+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:54:46.236036+0000","last_peered":"2023-10-08T09:54:46.236036+0000","last_clean":"2023-10-08T09:54:46.236036+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:54:46.236036+0000","last_undegraded":"2023-10-08T09:54:46.236036+0000","last_fullsized":"2023-10-08T09:54:46.236036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242680+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:46.242680+0000","last_peered":"2023-10-08T09:54:46.242680+0000","last_clean":"2023-10-08T09:54:46.242680+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:46.242680+0000","last_undegraded":"2023-10-08T09:54:46.242680+0000","last_fullsized":"2023-10-08T09:54:46.242680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":262,"num_read_kb":144,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.57799999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.879},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81200000000000006}]},{"osd":5,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.97999999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.86299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":1.03}]},{"osd":9,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.044},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.99399999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.071}]}]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1071183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.375},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60399999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64600000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001}]}]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":476213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.41499999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.44400000000000001,"15min":0.45800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.48699999999999999,"5min":0.746,"15min":0.746},"last":0.42399999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.59799999999999998,"15min":0.61299999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.752,"5min":1.339,"15min":1.339},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":1.125,"15min":1.125},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63800000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.502,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.48399999999999999,"15min":0.502},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.76600000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.65100000000000002,"15min":0.66500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79600000000000004,"5min":1.171,"15min":1.171},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.72799999999999998,"15min":0.747},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80300000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81499999999999995,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.749,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78200000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66900000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73599999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.39300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.86799999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.32500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.441}]},{"osd":3,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.57699999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.71799999999999997,"15min":0.69799999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.83699999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.96199999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89700000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67400000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.49199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.33100000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90800000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60199999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98399999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.501}]},{"osd":9,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91600000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57199999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63700000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27900000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55200000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52100000000000002,"5min":0.625,"15min":0.625},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.74099999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.624,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92000000000000004,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.78200000000000003,"15min":0.85599999999999998},"min":{"1min":0.39200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.71399999999999997,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57199999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.83699999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71599999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.746},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73299999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":479349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.53700000000000003,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.48899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67000000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.876}]},{"osd":6,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.022}]},{"osd":8,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.86199999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.91500000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89600000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":2.206,"15min":2.206},"last":0.498},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.627}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.374},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81299999999999994,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.628},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.503,"15min":0.51000000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65900000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56000000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45800000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.69699999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":476633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80700000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":2.698,"15min":2.698},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72799999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.052,"5min":0.625,"15min":0.53500000000000003},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.282,"5min":8.282,"15min":8.282},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.27600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":1.024,"15min":1.024},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.44500000000000001},"min":{"1min":0.34100000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.034,"15min":1.034},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.196,"5min":0.71499999999999997,"15min":0.629},"min":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.7949999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.751}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59099999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.79200000000000004,"15min":0.72199999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":1.718},{"interface":"front","average":{"1min":1.218,"5min":0.81200000000000006,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.1699999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.7310000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81399999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80000000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":475679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.501}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.755,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.55300000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64500000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63100000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60699999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65700000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498}]},{"osd":5,"last update":"Sun Oct 8 09:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70399999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.621},"min":{"1min":0.45500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.53000000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.443}]},{"osd":8,"last update":"Sun Oct 8 09:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.51300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93700000000000006,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.997,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.628},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1000000000000001,"5min":5.492,"15min":5.492},"last":0.72999999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.64100000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.748}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68300000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":1.089,"15min":1.089},"last":0.39300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86599999999999999,"5min":1.379,"15min":1.379},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":1.407,"15min":1.407},"last":0.37},{"interface":"front","average":{"1min":0.502,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":1.476,"15min":1.476},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91600000000000004,"5min":1.165,"15min":1.165},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98599999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55500000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98599999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95199999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.75}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:08.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T09:55:08.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:08 smithi097 ceph-mon[99491]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:08 smithi178 ceph-mon[103156]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: pgmap v298: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:08 smithi186 ceph-mon[104393]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 -- 172.21.15.97:0/1690506021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068b30 msgr2=0x7f21f010d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 --2- 172.21.15.97:0/1690506021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068b30 0x7f21f010d670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b0a0 tx=0x7f21e800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 -- 172.21.15.97:0/1690506021 shutdown_connections 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 --2- 172.21.15.97:0/1690506021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068b30 0x7f21f010d670 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 --2- 172.21.15.97:0/1690506021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f01067a0 0x7f21f0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 --2- 172.21.15.97:0/1690506021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f0068190 0x7f21f00685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 -- 172.21.15.97:0/1690506021 >> 172.21.15.97:0/1690506021 conn(0x7f21f0075790 msgr2=0x7f21f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 -- 172.21.15.97:0/1690506021 shutdown_connections 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.203+0000 7f21f66da700 1 -- 172.21.15.97:0/1690506021 wait complete. 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 Processor -- start 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 -- start start 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 0x7f21f019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 0x7f21f0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0197230 con 0x7f21f0068190 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f01973a0 con 0x7f21f0068b30 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f66da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0197560 con 0x7f21f01067a0 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:10.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56066/0 (socket says 172.21.15.97:56066) 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 -- 172.21.15.97:0/1226801698 learned_addr learned my addr 172.21.15.97:0/1226801698 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f4c77700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 0x7f21f0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21ef7fe700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 0x7f21f019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 -- 172.21.15.97:0/1226801698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 msgr2=0x7f21f0196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 0x7f21f0196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 -- 172.21.15.97:0/1226801698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 msgr2=0x7f21f019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 0x7f21f019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21effff700 1 -- 172.21.15.97:0/1226801698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009d00 con 0x7f21f0068190 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21ef7fe700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 0x7f21f019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.204+0000 7f21f4c77700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 0x7f21f0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21effff700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d980 tx=0x7f21e000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:10.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000bb60 con 0x7f21f0068190 2023-10-08T09:55:10.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f0197840 con 0x7f21f0068190 2023-10-08T09:55:10.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21e0007640 con 0x7f21f0068190 2023-10-08T09:55:10.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004e90 con 0x7f21f0068190 2023-10-08T09:55:10.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.205+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f01a3a30 con 0x7f21f0068190 2023-10-08T09:55:10.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.206+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f0108d50 con 0x7f21f0068190 2023-10-08T09:55:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.207+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f21e000bcc0 con 0x7f21f0068190 2023-10-08T09:55:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.207+0000 7f21ed7fa700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 0x7f21d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.208+0000 7f21ef7fe700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 0x7f21d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.208+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21e00cce30 con 0x7f21f0068190 2023-10-08T09:55:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.208+0000 7f21ef7fe700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 0x7f21d80a3940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.212+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f21e0096180 con 0x7f21f0068190 2023-10-08T09:55:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.397+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21f0197fd0 con 0x7f21d80a1480 2023-10-08T09:55:10.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.409+0000 7f21ed7fa700 1 -- 172.21.15.97:0/1226801698 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427459 (secure 0 0 0) 0x7f21f0197fd0 con 0x7f21d80a1480 2023-10-08T09:55:10.415 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 msgr2=0x7f21d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 0x7f21d80a3940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e4005ee0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 msgr2=0x7f21f019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d980 tx=0x7f21e000dd40 comp rx=0 tx=0).stop 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 shutdown_connections 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f21d80a1480 0x7f21d80a3940 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21f01067a0 0x7f21f0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f0068190 0x7f21f019ca40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 --2- 172.21.15.97:0/1226801698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0068b30 0x7f21f019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 >> 172.21.15.97:0/1226801698 conn(0x7f21f0075790 msgr2=0x7f21f00fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 shutdown_connections 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:10.413+0000 7f21f66da700 1 -- 172.21.15.97:0/1226801698 wait complete. 2023-10-08T09:55:10.419 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:10.925 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-08T09:55:09.896474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597820,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":388,"num_read_kb":253,"num_write":310,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337572,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604252,"statfs":{"total":1151940427776,"available":1151594754048,"internally_reserved":0,"allocated":13627392,"data_stored":7498200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003577"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242921+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:46.242921+0000","last_peered":"2023-10-08T09:54:46.242921+0000","last_clean":"2023-10-08T09:54:46.242921+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:46.242921+0000","last_undegraded":"2023-10-08T09:54:46.242921+0000","last_fullsized":"2023-10-08T09:54:46.242921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239846+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:46.239846+0000","last_peered":"2023-10-08T09:54:46.239846+0000","last_clean":"2023-10-08T09:54:46.239846+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:46.239846+0000","last_undegraded":"2023-10-08T09:54:46.239846+0000","last_fullsized":"2023-10-08T09:54:46.239846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240381+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:46.240381+0000","last_peered":"2023-10-08T09:54:46.240381+0000","last_clean":"2023-10-08T09:54:46.240381+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:46.240381+0000","last_undegraded":"2023-10-08T09:54:46.240381+0000","last_fullsized":"2023-10-08T09:54:46.240381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235313+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:46.235313+0000","last_peered":"2023-10-08T09:54:46.235313+0000","last_clean":"2023-10-08T09:54:46.235313+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:46.235313+0000","last_undegraded":"2023-10-08T09:54:46.235313+0000","last_fullsized":"2023-10-08T09:54:46.235313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237260+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:54:46.237260+0000","last_peered":"2023-10-08T09:54:46.237260+0000","last_clean":"2023-10-08T09:54:46.237260+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:54:46.237260+0000","last_undegraded":"2023-10-08T09:54:46.237260+0000","last_fullsized":"2023-10-08T09:54:46.237260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239591+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:46.239591+0000","last_peered":"2023-10-08T09:54:46.239591+0000","last_clean":"2023-10-08T09:54:46.239591+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:46.239591+0000","last_undegraded":"2023-10-08T09:54:46.239591+0000","last_fullsized":"2023-10-08T09:54:46.239591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239548+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:46.239548+0000","last_peered":"2023-10-08T09:54:46.239548+0000","last_clean":"2023-10-08T09:54:46.239548+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:46.239548+0000","last_undegraded":"2023-10-08T09:54:46.239548+0000","last_fullsized":"2023-10-08T09:54:46.239548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235925+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:54:46.235925+0000","last_peered":"2023-10-08T09:54:46.235925+0000","last_clean":"2023-10-08T09:54:46.235925+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:54:46.235925+0000","last_undegraded":"2023-10-08T09:54:46.235925+0000","last_fullsized":"2023-10-08T09:54:46.235925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242828+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:46.242828+0000","last_peered":"2023-10-08T09:54:46.242828+0000","last_clean":"2023-10-08T09:54:46.242828+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:46.242828+0000","last_undegraded":"2023-10-08T09:54:46.242828+0000","last_fullsized":"2023-10-08T09:54:46.242828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238442+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:46.238442+0000","last_peered":"2023-10-08T09:54:46.238442+0000","last_clean":"2023-10-08T09:54:46.238442+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:46.238442+0000","last_undegraded":"2023-10-08T09:54:46.238442+0000","last_fullsized":"2023-10-08T09:54:46.238442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239781+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:46.239781+0000","last_peered":"2023-10-08T09:54:46.239781+0000","last_clean":"2023-10-08T09:54:46.239781+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:46.239781+0000","last_undegraded":"2023-10-08T09:54:46.239781+0000","last_fullsized":"2023-10-08T09:54:46.239781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372615+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:54:47.372615+0000","last_peered":"2023-10-08T09:54:47.372615+0000","last_clean":"2023-10-08T09:54:47.372615+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:54:47.372615+0000","last_undegraded":"2023-10-08T09:54:47.372615+0000","last_fullsized":"2023-10-08T09:54:47.372615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238188+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:46.238188+0000","last_peered":"2023-10-08T09:54:46.238188+0000","last_clean":"2023-10-08T09:54:46.238188+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:46.238188+0000","last_undegraded":"2023-10-08T09:54:46.238188+0000","last_fullsized":"2023-10-08T09:54:46.238188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235877+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:46.235877+0000","last_peered":"2023-10-08T09:54:46.235877+0000","last_clean":"2023-10-08T09:54:46.235877+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:46.235877+0000","last_undegraded":"2023-10-08T09:54:46.235877+0000","last_fullsized":"2023-10-08T09:54:46.235877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235918+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:46.235918+0000","last_peered":"2023-10-08T09:54:46.235918+0000","last_clean":"2023-10-08T09:54:46.235918+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:46.235918+0000","last_undegraded":"2023-10-08T09:54:46.235918+0000","last_fullsized":"2023-10-08T09:54:46.235918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241439+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:46.241439+0000","last_peered":"2023-10-08T09:54:46.241439+0000","last_clean":"2023-10-08T09:54:46.241439+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:46.241439+0000","last_undegraded":"2023-10-08T09:54:46.241439+0000","last_fullsized":"2023-10-08T09:54:46.241439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236163+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:54:46.236163+0000","last_peered":"2023-10-08T09:54:46.236163+0000","last_clean":"2023-10-08T09:54:46.236163+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:54:46.236163+0000","last_undegraded":"2023-10-08T09:54:46.236163+0000","last_fullsized":"2023-10-08T09:54:46.236163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239642+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:46.239642+0000","last_peered":"2023-10-08T09:54:46.239642+0000","last_clean":"2023-10-08T09:54:46.239642+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:46.239642+0000","last_undegraded":"2023-10-08T09:54:46.239642+0000","last_fullsized":"2023-10-08T09:54:46.239642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:03.249827+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:03.249827+0000","last_peered":"2023-10-08T09:55:03.249827+0000","last_clean":"2023-10-08T09:55:03.249827+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:03.249827+0000","last_undegraded":"2023-10-08T09:55:03.249827+0000","last_fullsized":"2023-10-08T09:55:03.249827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239716+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:46.239716+0000","last_peered":"2023-10-08T09:54:46.239716+0000","last_clean":"2023-10-08T09:54:46.239716+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:46.239716+0000","last_undegraded":"2023-10-08T09:54:46.239716+0000","last_fullsized":"2023-10-08T09:54:46.239716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242686+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:46.242686+0000","last_peered":"2023-10-08T09:54:46.242686+0000","last_clean":"2023-10-08T09:54:46.242686+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:46.242686+0000","last_undegraded":"2023-10-08T09:54:46.242686+0000","last_fullsized":"2023-10-08T09:54:46.242686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235223+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:46.235223+0000","last_peered":"2023-10-08T09:54:46.235223+0000","last_clean":"2023-10-08T09:54:46.235223+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:46.235223+0000","last_undegraded":"2023-10-08T09:54:46.235223+0000","last_fullsized":"2023-10-08T09:54:46.235223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239764+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:46.239764+0000","last_peered":"2023-10-08T09:54:46.239764+0000","last_clean":"2023-10-08T09:54:46.239764+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:46.239764+0000","last_undegraded":"2023-10-08T09:54:46.239764+0000","last_fullsized":"2023-10-08T09:54:46.239764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237422+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:46.237422+0000","last_peered":"2023-10-08T09:54:46.237422+0000","last_clean":"2023-10-08T09:54:46.237422+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:46.237422+0000","last_undegraded":"2023-10-08T09:54:46.237422+0000","last_fullsized":"2023-10-08T09:54:46.237422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235537+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:46.235537+0000","last_peered":"2023-10-08T09:54:46.235537+0000","last_clean":"2023-10-08T09:54:46.235537+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:46.235537+0000","last_undegraded":"2023-10-08T09:54:46.235537+0000","last_fullsized":"2023-10-08T09:54:46.235537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:04.250636+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:04.250636+0000","last_peered":"2023-10-08T09:55:04.250636+0000","last_clean":"2023-10-08T09:55:04.250636+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:04.250636+0000","last_undegraded":"2023-10-08T09:55:04.250636+0000","last_fullsized":"2023-10-08T09:55:04.250636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236359+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:54:46.236359+0000","last_peered":"2023-10-08T09:54:46.236359+0000","last_clean":"2023-10-08T09:54:46.236359+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:54:46.236359+0000","last_undegraded":"2023-10-08T09:54:46.236359+0000","last_fullsized":"2023-10-08T09:54:46.236359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237184+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:46.237184+0000","last_peered":"2023-10-08T09:54:46.237184+0000","last_clean":"2023-10-08T09:54:46.237184+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:46.237184+0000","last_undegraded":"2023-10-08T09:54:46.237184+0000","last_fullsized":"2023-10-08T09:54:46.237184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241365+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:46.241365+0000","last_peered":"2023-10-08T09:54:46.241365+0000","last_clean":"2023-10-08T09:54:46.241365+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:46.241365+0000","last_undegraded":"2023-10-08T09:54:46.241365+0000","last_fullsized":"2023-10-08T09:54:46.241365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236355+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:54:46.236355+0000","last_peered":"2023-10-08T09:54:46.236355+0000","last_clean":"2023-10-08T09:54:46.236355+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:54:46.236355+0000","last_undegraded":"2023-10-08T09:54:46.236355+0000","last_fullsized":"2023-10-08T09:54:46.236355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237737+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:46.237737+0000","last_peered":"2023-10-08T09:54:46.237737+0000","last_clean":"2023-10-08T09:54:46.237737+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:46.237737+0000","last_undegraded":"2023-10-08T09:54:46.237737+0000","last_fullsized":"2023-10-08T09:54:46.237737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'12","reported_seq":137,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:05.273176+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:05.273176+0000","last_peered":"2023-10-08T09:55:05.273176+0000","last_clean":"2023-10-08T09:55:05.273176+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:05.273176+0000","last_undegraded":"2023-10-08T09:55:05.273176+0000","last_fullsized":"2023-10-08T09:55:05.273176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":44,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240097+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:46.240097+0000","last_peered":"2023-10-08T09:54:46.240097+0000","last_clean":"2023-10-08T09:54:46.240097+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:46.240097+0000","last_undegraded":"2023-10-08T09:54:46.240097+0000","last_fullsized":"2023-10-08T09:54:46.240097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242488+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:46.242488+0000","last_peered":"2023-10-08T09:54:46.242488+0000","last_clean":"2023-10-08T09:54:46.242488+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:46.242488+0000","last_undegraded":"2023-10-08T09:54:46.242488+0000","last_fullsized":"2023-10-08T09:54:46.242488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238020+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:46.238020+0000","last_peered":"2023-10-08T09:54:46.238020+0000","last_clean":"2023-10-08T09:54:46.238020+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:46.238020+0000","last_undegraded":"2023-10-08T09:54:46.238020+0000","last_fullsized":"2023-10-08T09:54:46.238020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:03.249309+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:03.249309+0000","last_peered":"2023-10-08T09:55:03.249309+0000","last_clean":"2023-10-08T09:55:03.249309+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:03.249309+0000","last_undegraded":"2023-10-08T09:55:03.249309+0000","last_fullsized":"2023-10-08T09:55:03.249309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.267960+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.267960+0000","last_peered":"2023-10-08T09:54:47.267960+0000","last_clean":"2023-10-08T09:54:47.267960+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.267960+0000","last_undegraded":"2023-10-08T09:54:47.267960+0000","last_fullsized":"2023-10-08T09:54:47.267960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242804+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:46.242804+0000","last_peered":"2023-10-08T09:54:46.242804+0000","last_clean":"2023-10-08T09:54:46.242804+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:46.242804+0000","last_undegraded":"2023-10-08T09:54:46.242804+0000","last_fullsized":"2023-10-08T09:54:46.242804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237811+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:46.237811+0000","last_peered":"2023-10-08T09:54:46.237811+0000","last_clean":"2023-10-08T09:54:46.237811+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:46.237811+0000","last_undegraded":"2023-10-08T09:54:46.237811+0000","last_fullsized":"2023-10-08T09:54:46.237811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:04.249955+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:04.249955+0000","last_peered":"2023-10-08T09:55:04.249955+0000","last_clean":"2023-10-08T09:55:04.249955+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:04.249955+0000","last_undegraded":"2023-10-08T09:55:04.249955+0000","last_fullsized":"2023-10-08T09:55:04.249955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235354+0000","last_change":"2023-10-08T09:53:56.887772+0000","last_active":"2023-10-08T09:54:46.235354+0000","last_peered":"2023-10-08T09:54:46.235354+0000","last_clean":"2023-10-08T09:54:46.235354+0000","last_became_active":"2023-10-08T09:53:56.887155+0000","last_became_peered":"2023-10-08T09:53:56.887155+0000","last_unstale":"2023-10-08T09:54:46.235354+0000","last_undegraded":"2023-10-08T09:54:46.235354+0000","last_fullsized":"2023-10-08T09:54:46.235354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:45.184494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242732+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:46.242732+0000","last_peered":"2023-10-08T09:54:46.242732+0000","last_clean":"2023-10-08T09:54:46.242732+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:46.242732+0000","last_undegraded":"2023-10-08T09:54:46.242732+0000","last_fullsized":"2023-10-08T09:54:46.242732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264240+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264240+0000","last_peered":"2023-10-08T09:54:47.264240+0000","last_clean":"2023-10-08T09:54:47.264240+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264240+0000","last_undegraded":"2023-10-08T09:54:47.264240+0000","last_fullsized":"2023-10-08T09:54:47.264240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235224+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:46.235224+0000","last_peered":"2023-10-08T09:54:46.235224+0000","last_clean":"2023-10-08T09:54:46.235224+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:46.235224+0000","last_undegraded":"2023-10-08T09:54:46.235224+0000","last_fullsized":"2023-10-08T09:54:46.235224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238500+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:46.238500+0000","last_peered":"2023-10-08T09:54:46.238500+0000","last_clean":"2023-10-08T09:54:46.238500+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:46.238500+0000","last_undegraded":"2023-10-08T09:54:46.238500+0000","last_fullsized":"2023-10-08T09:54:46.238500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235824+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:46.235824+0000","last_peered":"2023-10-08T09:54:46.235824+0000","last_clean":"2023-10-08T09:54:46.235824+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:46.235824+0000","last_undegraded":"2023-10-08T09:54:46.235824+0000","last_fullsized":"2023-10-08T09:54:46.235824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239283+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:46.239283+0000","last_peered":"2023-10-08T09:54:46.239283+0000","last_clean":"2023-10-08T09:54:46.239283+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:46.239283+0000","last_undegraded":"2023-10-08T09:54:46.239283+0000","last_fullsized":"2023-10-08T09:54:46.239283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241730+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:46.241730+0000","last_peered":"2023-10-08T09:54:46.241730+0000","last_clean":"2023-10-08T09:54:46.241730+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:46.241730+0000","last_undegraded":"2023-10-08T09:54:46.241730+0000","last_fullsized":"2023-10-08T09:54:46.241730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235841+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:54:46.235841+0000","last_peered":"2023-10-08T09:54:46.235841+0000","last_clean":"2023-10-08T09:54:46.235841+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:54:46.235841+0000","last_undegraded":"2023-10-08T09:54:46.235841+0000","last_fullsized":"2023-10-08T09:54:46.235841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235382+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:46.235382+0000","last_peered":"2023-10-08T09:54:46.235382+0000","last_clean":"2023-10-08T09:54:46.235382+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:46.235382+0000","last_undegraded":"2023-10-08T09:54:46.235382+0000","last_fullsized":"2023-10-08T09:54:46.235382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242474+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:46.242474+0000","last_peered":"2023-10-08T09:54:46.242474+0000","last_clean":"2023-10-08T09:54:46.242474+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:46.242474+0000","last_undegraded":"2023-10-08T09:54:46.242474+0000","last_fullsized":"2023-10-08T09:54:46.242474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239156+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:46.239156+0000","last_peered":"2023-10-08T09:54:46.239156+0000","last_clean":"2023-10-08T09:54:46.239156+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:46.239156+0000","last_undegraded":"2023-10-08T09:54:46.239156+0000","last_fullsized":"2023-10-08T09:54:46.239156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235543+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:46.235543+0000","last_peered":"2023-10-08T09:54:46.235543+0000","last_clean":"2023-10-08T09:54:46.235543+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:46.235543+0000","last_undegraded":"2023-10-08T09:54:46.235543+0000","last_fullsized":"2023-10-08T09:54:46.235543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235875+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:54:46.235875+0000","last_peered":"2023-10-08T09:54:46.235875+0000","last_clean":"2023-10-08T09:54:46.235875+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:54:46.235875+0000","last_undegraded":"2023-10-08T09:54:46.235875+0000","last_fullsized":"2023-10-08T09:54:46.235875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240249+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:46.240249+0000","last_peered":"2023-10-08T09:54:46.240249+0000","last_clean":"2023-10-08T09:54:46.240249+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:46.240249+0000","last_undegraded":"2023-10-08T09:54:46.240249+0000","last_fullsized":"2023-10-08T09:54:46.240249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239715+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:46.239715+0000","last_peered":"2023-10-08T09:54:46.239715+0000","last_clean":"2023-10-08T09:54:46.239715+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:46.239715+0000","last_undegraded":"2023-10-08T09:54:46.239715+0000","last_fullsized":"2023-10-08T09:54:46.239715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242966+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:46.242966+0000","last_peered":"2023-10-08T09:54:46.242966+0000","last_clean":"2023-10-08T09:54:46.242966+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:46.242966+0000","last_undegraded":"2023-10-08T09:54:46.242966+0000","last_fullsized":"2023-10-08T09:54:46.242966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239466+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:46.239466+0000","last_peered":"2023-10-08T09:54:46.239466+0000","last_clean":"2023-10-08T09:54:46.239466+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:46.239466+0000","last_undegraded":"2023-10-08T09:54:46.239466+0000","last_fullsized":"2023-10-08T09:54:46.239466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239080+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:46.239080+0000","last_peered":"2023-10-08T09:54:46.239080+0000","last_clean":"2023-10-08T09:54:46.239080+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:46.239080+0000","last_undegraded":"2023-10-08T09:54:46.239080+0000","last_fullsized":"2023-10-08T09:54:46.239080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239994+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:46.239994+0000","last_peered":"2023-10-08T09:54:46.239994+0000","last_clean":"2023-10-08T09:54:46.239994+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:46.239994+0000","last_undegraded":"2023-10-08T09:54:46.239994+0000","last_fullsized":"2023-10-08T09:54:46.239994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243338+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:46.243338+0000","last_peered":"2023-10-08T09:54:46.243338+0000","last_clean":"2023-10-08T09:54:46.243338+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:46.243338+0000","last_undegraded":"2023-10-08T09:54:46.243338+0000","last_fullsized":"2023-10-08T09:54:46.243338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239967+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:46.239967+0000","last_peered":"2023-10-08T09:54:46.239967+0000","last_clean":"2023-10-08T09:54:46.239967+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:46.239967+0000","last_undegraded":"2023-10-08T09:54:46.239967+0000","last_fullsized":"2023-10-08T09:54:46.239967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235937+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:46.235937+0000","last_peered":"2023-10-08T09:54:46.235937+0000","last_clean":"2023-10-08T09:54:46.235937+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:46.235937+0000","last_undegraded":"2023-10-08T09:54:46.235937+0000","last_fullsized":"2023-10-08T09:54:46.235937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235296+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:46.235296+0000","last_peered":"2023-10-08T09:54:46.235296+0000","last_clean":"2023-10-08T09:54:46.235296+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:46.235296+0000","last_undegraded":"2023-10-08T09:54:46.235296+0000","last_fullsized":"2023-10-08T09:54:46.235296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.236036+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:54:46.236036+0000","last_peered":"2023-10-08T09:54:46.236036+0000","last_clean":"2023-10-08T09:54:46.236036+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:54:46.236036+0000","last_undegraded":"2023-10-08T09:54:46.236036+0000","last_fullsized":"2023-10-08T09:54:46.236036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242680+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:46.242680+0000","last_peered":"2023-10-08T09:54:46.242680+0000","last_clean":"2023-10-08T09:54:46.242680+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:46.242680+0000","last_undegraded":"2023-10-08T09:54:46.242680+0000","last_fullsized":"2023-10-08T09:54:46.242680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":262,"num_read_kb":144,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":22356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.57799999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.879},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81200000000000006}]},{"osd":5,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.97999999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.86299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":1.03}]},{"osd":9,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.044},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.99399999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.071}]}]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1071183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.375},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.5}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60399999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64600000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75800000000000001}]}]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":476213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.41499999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.44400000000000001,"15min":0.45800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.48699999999999999,"5min":0.746,"15min":0.746},"last":0.42399999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.59799999999999998,"15min":0.61299999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.752,"5min":1.339,"15min":1.339},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.65100000000000002,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.73399999999999999,"5min":1.125,"15min":1.125},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63800000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.502,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.48399999999999999,"15min":0.502},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.58599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.76600000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.65100000000000002,"15min":0.66500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79600000000000004,"5min":1.171,"15min":1.171},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.72799999999999998,"15min":0.747},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80300000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81499999999999995,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.749,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78600000000000003,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78200000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66900000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73599999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.52500000000000002,"15min":0.505},"min":{"1min":0.39300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.86799999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.44800000000000001},"min":{"1min":0.32500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.441}]},{"osd":3,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.57699999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.71799999999999997,"15min":0.69799999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.83699999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.96199999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.89700000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.43099999999999999},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67400000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.49199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61399999999999999,"15min":0.60699999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.33100000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90800000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60199999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98399999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.501}]},{"osd":9,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.91600000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57199999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63700000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27900000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55200000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38100000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52100000000000002,"5min":0.625,"15min":0.625},"last":0.39800000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.74099999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.44600000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.624,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92000000000000004,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.78200000000000003,"15min":0.85599999999999998},"min":{"1min":0.39200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.71399999999999997,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57199999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.83699999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85299999999999998,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71599999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.746},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73299999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":479349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.53700000000000003,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.48899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67000000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.876}]},{"osd":6,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.022}]},{"osd":8,"last update":"Sun Oct 8 09:54:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.57599999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.86199999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:54:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.91500000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.89600000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.49399999999999999},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80300000000000005,"5min":2.206,"15min":2.206},"last":0.498},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.627}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.374},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81299999999999994,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.629,"15min":0.628},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.503,"15min":0.51000000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65900000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56000000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.624,"15min":0.63500000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45800000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.69699999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":476633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80700000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.751},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.441,"5min":0.318,"15min":0.318},"max":{"1min":0.84599999999999997,"5min":2.698,"15min":2.698},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72799999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.052,"5min":0.625,"15min":0.53500000000000003},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":8.282,"5min":8.282,"15min":8.282},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.27600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72399999999999998,"5min":1.024,"15min":1.024},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.44500000000000001},"min":{"1min":0.34100000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.034,"15min":1.034},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.196,"5min":0.71499999999999997,"15min":0.629},"min":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.7949999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76900000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81200000000000006}]},{"osd":9,"last update":"Sun Oct 8 09:54:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":0.79200000000000004,"15min":0.72199999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":8.2279999999999998,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.218,"5min":0.81200000000000006,"15min":0.74399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.1699999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82399999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.91200000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":475679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.27700000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.623},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.755,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.32900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.375}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53700000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52800000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55100000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51700000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498}]},{"osd":5,"last update":"Sun Oct 8 09:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70399999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.621},"min":{"1min":0.45500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.53000000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.443}]},{"osd":8,"last update":"Sun Oct 8 09:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.51300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93700000000000006,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.53000000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.997,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.628},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":0.96799999999999997},"min":{"1min":0.64800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1000000000000001,"5min":5.492,"15min":5.492},"last":0.72999999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.64100000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.748}]}]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68300000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79700000000000004,"5min":1.089,"15min":1.089},"last":0.39300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86599999999999999,"5min":1.379,"15min":1.379},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.30499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":1.407,"15min":1.407},"last":0.37},{"interface":"front","average":{"1min":0.502,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.29399999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.48299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":1.476,"15min":1.476},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91900000000000004,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91600000000000004,"5min":1.165,"15min":1.165},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98599999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55500000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98599999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95199999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.75}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:10.937 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-08T09:55:10.937 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T09:55:10.938 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-08T09:55:10.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-08T09:55:11.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:11 smithi097 ceph-mon[99491]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:11 smithi178 ceph-mon[103156]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:11.501 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:11 smithi186 ceph-mon[104393]: pgmap v299: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:12 smithi097 ceph-mon[99491]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:12 smithi178 ceph-mon[103156]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:12 smithi186 ceph-mon[104393]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:12.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/2439507214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 msgr2=0x7fe1f00685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:12.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/2439507214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f00685f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009990 tx=0x7fe1dc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/2439507214 shutdown_connections 2023-10-08T09:55:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/2439507214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 0x7fe1f010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/2439507214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 0x7fe1f0106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/2439507214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f00685f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/2439507214 >> 172.21.15.97:0/2439507214 conn(0x7fe1f0075790 msgr2=0x7fe1f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/2439507214 shutdown_connections 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.806+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/2439507214 wait complete. 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 Processor -- start 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 -- start start 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 0x7fe1f019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 0x7fe1f0196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01970c0 con 0x7fe1f0068190 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0197230 con 0x7fe1f0068b30 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f6fb3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01973f0 con 0x7fe1f01067c0 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56078/0 (socket says 172.21.15.97:56078) 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 -- 172.21.15.97:0/353592025 learned_addr learned my addr 172.21.15.97:0/353592025 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1effff700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 0x7fe1f019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f5550700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 0x7fe1f0196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 -- 172.21.15.97:0/353592025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 msgr2=0x7fe1f0196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 0x7fe1f0196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 -- 172.21.15.97:0/353592025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 msgr2=0x7fe1f019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.807+0000 7fe1f4d4f700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 0x7fe1f019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1f4d4f700 1 -- 172.21.15.97:0/353592025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0095f0 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1f4d4f700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400e9a0 tx=0x7fe1e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4009790 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1e400cbd0 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f01976d0 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4018680 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.808+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f01a38c0 con 0x7fe1f0068190 2023-10-08T09:55:12.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.809+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f0108de0 con 0x7fe1f0068190 2023-10-08T09:55:12.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.810+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe1e400cd40 con 0x7fe1f0068190 2023-10-08T09:55:12.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.811+0000 7fe1edffb700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 0x7fe1e00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:12.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.811+0000 7fe1effff700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 0x7fe1e00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:12.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.811+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe1e4014070 con 0x7fe1f0068190 2023-10-08T09:55:12.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.811+0000 7fe1effff700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 0x7fe1e00a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b2e0 tx=0x7fe1dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:12.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:12.815+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1e4052100 con 0x7fe1f0068190 2023-10-08T09:55:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.047+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe1f0197ed0 con 0x7fe1f0068190 2023-10-08T09:55:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.047+0000 7fe1edffb700 1 -- 172.21.15.97:0/353592025 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe1e40964d0 con 0x7fe1f0068190 2023-10-08T09:55:13.048 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:13.049 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 msgr2=0x7fe1e00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 0x7fe1e00a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b2e0 tx=0x7fe1dc005960 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 msgr2=0x7fe1f019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400e9a0 tx=0x7fe1e400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 shutdown_connections 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe1e00a1500 0x7fe1e00a39c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1f01067c0 0x7fe1f0196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1f0068190 0x7fe1f019c8d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 --2- 172.21.15.97:0/353592025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f0068b30 0x7fe1f019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 >> 172.21.15.97:0/353592025 conn(0x7fe1f0075790 msgr2=0x7fe1f00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:13.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 shutdown_connections 2023-10-08T09:55:13.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.050+0000 7fe1f6fb3700 1 -- 172.21.15.97:0/353592025 wait complete. 2023-10-08T09:55:13.450 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:13 smithi097 ceph-mon[99491]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:13.451 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:13 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/353592025' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:55:13.452 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-08T09:55:13.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:13 smithi178 ceph-mon[103156]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:13 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/353592025' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:55:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:13 smithi186 ceph-mon[104393]: pgmap v300: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:13 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/353592025' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:55:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 -- 172.21.15.97:0/1513407195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590100b40 msgr2=0x7fa590100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 --2- 172.21.15.97:0/1513407195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590100b40 0x7fa590100f60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa580009990 tx=0x7fa580009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 -- 172.21.15.97:0/1513407195 shutdown_connections 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 --2- 172.21.15.97:0/1513407195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 0x7fa590107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 --2- 172.21.15.97:0/1513407195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590100b40 0x7fa590100f60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 --2- 172.21.15.97:0/1513407195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5900ff870 0x7fa5900ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 -- 172.21.15.97:0/1513407195 >> 172.21.15.97:0/1513407195 conn(0x7fa5900fa2c0 msgr2=0x7fa5900fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 -- 172.21.15.97:0/1513407195 shutdown_connections 2023-10-08T09:55:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.579+0000 7fa5987e6700 1 -- 172.21.15.97:0/1513407195 wait complete. 2023-10-08T09:55:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 Processor -- start 2023-10-08T09:55:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 -- start start 2023-10-08T09:55:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 0x7fa590198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 0x7fa59019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901a0380 con 0x7fa5900ff870 2023-10-08T09:55:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901a04c0 con 0x7fa590100b40 2023-10-08T09:55:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa5987e6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901a0650 con 0x7fa590100210 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa596582700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa596582700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56092/0 (socket says 172.21.15.97:56092) 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa596582700 1 -- 172.21.15.97:0/3219959473 learned_addr learned my addr 172.21.15.97:0/3219959473 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa596d83700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 0x7fa59019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.580+0000 7fa595d81700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 0x7fa590198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 -- 172.21.15.97:0/3219959473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 msgr2=0x7fa590198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 0x7fa590198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 -- 172.21.15.97:0/3219959473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 msgr2=0x7fa59019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 0x7fa59019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 -- 172.21.15.97:0/3219959473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5800095f0 con 0x7fa5900ff870 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596d83700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 0x7fa59019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:13.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa596582700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa5800095c0 tx=0x7fa58000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58001d070 con 0x7fa5900ff870 2023-10-08T09:55:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa580003960 con 0x7fa5900ff870 2023-10-08T09:55:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5901a08d0 con 0x7fa5900ff870 2023-10-08T09:55:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580005650 con 0x7fa5900ff870 2023-10-08T09:55:13.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.581+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a0cf0 con 0x7fa5900ff870 2023-10-08T09:55:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.582+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa590066bc0 con 0x7fa5900ff870 2023-10-08T09:55:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.583+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa58000fad0 con 0x7fa5900ff870 2023-10-08T09:55:13.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.584+0000 7fa5877fe700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 0x7fa57c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.584+0000 7fa595d81700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 0x7fa57c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.584+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5800ce100 con 0x7fa5900ff870 2023-10-08T09:55:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.584+0000 7fa595d81700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 0x7fa57c0a3940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005950 tx=0x7fa58c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.588+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5800974e0 con 0x7fa5900ff870 2023-10-08T09:55:13.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.781+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5901a1020 con 0x7fa5900ff870 2023-10-08T09:55:13.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.783+0000 7fa5877fe700 1 -- 172.21.15.97:0/3219959473 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7fa58009ab00 con 0x7fa5900ff870 2023-10-08T09:55:13.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:13.784 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 msgr2=0x7fa57c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 0x7fa57c0a3940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005950 tx=0x7fa58c00b410 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 msgr2=0x7fa590198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa5800095c0 tx=0x7fa58000f690 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 shutdown_connections 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa57c0a1480 0x7fa57c0a3940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590100b40 0x7fa59019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5900ff870 0x7fa590198850 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 --2- 172.21.15.97:0/3219959473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa590100210 0x7fa590198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 >> 172.21.15.97:0/3219959473 conn(0x7fa5900fa2c0 msgr2=0x7fa5900fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:13.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 shutdown_connections 2023-10-08T09:55:13.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.785+0000 7fa5987e6700 1 -- 172.21.15.97:0/3219959473 wait complete. 2023-10-08T09:55:13.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:13.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- 172.21.15.97:0/1431885807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100260 msgr2=0x7efd30107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- 172.21.15.97:0/1431885807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100260 0x7efd30107b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b0a0 tx=0x7efd2800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- 172.21.15.97:0/1431885807 shutdown_connections 2023-10-08T09:55:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- 172.21.15.97:0/1431885807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100260 0x7efd30107b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- 172.21.15.97:0/1431885807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd30100c50 0x7efd30101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- 172.21.15.97:0/1431885807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd300ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- 172.21.15.97:0/1431885807 >> 172.21.15.97:0/1431885807 conn(0x7efd300fa390 msgr2=0x7efd300fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- 172.21.15.97:0/1431885807 shutdown_connections 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- 172.21.15.97:0/1431885807 wait complete. 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 Processor -- start 2023-10-08T09:55:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- start start 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd301035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd30100260 0x7efd30101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301028d0 con 0x7efd30100c50 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd30102a40 con 0x7efd30100260 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.923+0000 7efd37b89700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301a0840 con 0x7efd300ff8a0 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56110/0 (socket says 172.21.15.97:56110) 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 -- 172.21.15.97:0/452368154 learned_addr learned my addr 172.21.15.97:0/452368154 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd35925700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd301035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 -- 172.21.15.97:0/452368154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 msgr2=0x7efd301035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd301035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 -- 172.21.15.97:0/452368154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd30100260 msgr2=0x7efd30101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd30100260 0x7efd30101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 -- 172.21.15.97:0/452368154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009d00 con 0x7efd30100c50 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd35925700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd301035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd36126700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd28006070 tx=0x7efd28004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800e040 con 0x7efd30100c50 2023-10-08T09:55:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2801db50 con 0x7efd30100c50 2023-10-08T09:55:13.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd301a0a40 con 0x7efd30100c50 2023-10-08T09:55:13.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2801bbd0 con 0x7efd30100c50 2023-10-08T09:55:13.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.924+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd301a0ec0 con 0x7efd30100c50 2023-10-08T09:55:13.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.925+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30066ca0 con 0x7efd30100c50 2023-10-08T09:55:13.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.926+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efd2801bd30 con 0x7efd30100c50 2023-10-08T09:55:13.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.927+0000 7efd26ffd700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 0x7efd1c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:13.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.927+0000 7efd35925700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 0x7efd1c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:13.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.927+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efd280cf450 con 0x7efd30100c50 2023-10-08T09:55:13.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.927+0000 7efd35925700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 0x7efd1c0a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:13.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:13.930+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd28098850 con 0x7efd30100c50 2023-10-08T09:55:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.124+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd3010b020 con 0x7efd30100c50 2023-10-08T09:55:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.124+0000 7efd26ffd700 1 -- 172.21.15.97:0/452368154 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd280cf020 con 0x7efd30100c50 2023-10-08T09:55:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.126+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 msgr2=0x7efd1c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.126+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 0x7efd1c0a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20005e30 comp rx=0 tx=0).stop 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.126+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 msgr2=0x7efd301021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.126+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efd28006070 tx=0x7efd28004db0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 shutdown_connections 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efd1c0a1480 0x7efd1c0a3940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30100c50 0x7efd301021d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd300ff8a0 0x7efd301035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 --2- 172.21.15.97:0/452368154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd30100260 0x7efd30101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 >> 172.21.15.97:0/452368154 conn(0x7efd300fa390 msgr2=0x7efd300fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 shutdown_connections 2023-10-08T09:55:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.127+0000 7efd37b89700 1 -- 172.21.15.97:0/452368154 wait complete. 2023-10-08T09:55:14.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 -- 172.21.15.97:0/3272031324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940fff70 msgr2=0x7f89941003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 --2- 172.21.15.97:0/3272031324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940fff70 0x7f89941003f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8984009990 tx=0x7f8984009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 -- 172.21.15.97:0/3272031324 shutdown_connections 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 --2- 172.21.15.97:0/3272031324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8994100930 0x7f8994107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 --2- 172.21.15.97:0/3272031324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 0x7f89940ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 --2- 172.21.15.97:0/3272031324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940fff70 0x7f89941003f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 -- 172.21.15.97:0/3272031324 >> 172.21.15.97:0/3272031324 conn(0x7f89940fa390 msgr2=0x7f89940fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 -- 172.21.15.97:0/3272031324 shutdown_connections 2023-10-08T09:55:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.264+0000 7f8999f16700 1 -- 172.21.15.97:0/3272031324 wait complete. 2023-10-08T09:55:14.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 Processor -- start 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 -- start start 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 0x7f89941955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89940fff70 0x7f8994195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899419d100 con 0x7f8994100930 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899419d240 con 0x7f89940fff70 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8999f16700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899419d3d0 con 0x7f89940fee20 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56124/0 (socket says 172.21.15.97:56124) 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 -- 172.21.15.97:0/530875010 learned_addr learned my addr 172.21.15.97:0/530875010 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8992ffd700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89940fff70 0x7f8994195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f89937fe700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 0x7f89941955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 -- 172.21.15.97:0/530875010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 msgr2=0x7f89941955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 0x7f89941955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 -- 172.21.15.97:0/530875010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89940fff70 msgr2=0x7f8994195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.265+0000 7f8993fff700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89940fff70 0x7f8994195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8993fff700 1 -- 172.21.15.97:0/530875010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840095f0 con 0x7f8994100930 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8993fff700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f898c007ee0 tx=0x7f898c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c017070 con 0x7f8994100930 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f898c00f040 con 0x7f8994100930 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899419d6b0 con 0x7f8994100930 2023-10-08T09:55:14.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c005580 con 0x7f8994100930 2023-10-08T09:55:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.266+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899419dbc0 con 0x7f8994100930 2023-10-08T09:55:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.267+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8994106240 con 0x7f8994100930 2023-10-08T09:55:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.268+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f898c00db20 con 0x7f8994100930 2023-10-08T09:55:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.268+0000 7f8990ff9700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 0x7f897c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.269+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f898c0ccd80 con 0x7f8994100930 2023-10-08T09:55:14.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.269+0000 7f89937fe700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 0x7f897c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.269+0000 7f89937fe700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 0x7f897c0a3940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8988005950 tx=0x7f898800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.272+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f898c0960d0 con 0x7f8994100930 2023-10-08T09:55:14.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:14 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3219959473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:14.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:14 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/452368154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:14 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3219959473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:14.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:14 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/452368154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.464+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f899410b130 con 0x7f8994100930 2023-10-08T09:55:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.464+0000 7f8990ff9700 1 -- 172.21.15.97:0/530875010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f898c0996f0 con 0x7f8994100930 2023-10-08T09:55:14.465 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 msgr2=0x7f897c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 0x7f897c0a3940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8988005950 tx=0x7f898800a300 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 msgr2=0x7f899419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f898c007ee0 tx=0x7f898c00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 shutdown_connections 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f897c0a1480 0x7f897c0a3940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8994100930 0x7f899419cb30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89940fee20 0x7f89941955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 --2- 172.21.15.97:0/530875010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89940fff70 0x7f8994195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 >> 172.21.15.97:0/530875010 conn(0x7f89940fa390 msgr2=0x7f8994103380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 shutdown_connections 2023-10-08T09:55:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.467+0000 7f8999f16700 1 -- 172.21.15.97:0/530875010 wait complete. 2023-10-08T09:55:14.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:14 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3219959473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:14.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:14 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/452368154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:14.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 -- 172.21.15.97:0/2445062009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c0101240 msgr2=0x7f82c0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 --2- 172.21.15.97:0/2445062009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c0101240 0x7f82c0107af0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b0d0 tx=0x7f82b800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 -- 172.21.15.97:0/2445062009 shutdown_connections 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 --2- 172.21.15.97:0/2445062009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c0101240 0x7f82c0107af0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 --2- 172.21.15.97:0/2445062009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c00ff6b0 0x7f82c00ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 --2- 172.21.15.97:0/2445062009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.609+0000 7f82c7c49700 1 -- 172.21.15.97:0/2445062009 >> 172.21.15.97:0/2445062009 conn(0x7f82c00fac40 msgr2=0x7f82c00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.610+0000 7f82c7c49700 1 -- 172.21.15.97:0/2445062009 shutdown_connections 2023-10-08T09:55:14.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.610+0000 7f82c7c49700 1 -- 172.21.15.97:0/2445062009 wait complete. 2023-10-08T09:55:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.610+0000 7f82c7c49700 1 Processor -- start 2023-10-08T09:55:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.610+0000 7f82c7c49700 1 -- start start 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c0101240 0x7f82c019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019e2f0 con 0x7f82c00ff6b0 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019e460 con 0x7f82c0101240 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019e620 con 0x7f82c01008a0 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56134/0 (socket says 172.21.15.97:56134) 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 -- 172.21.15.97:0/670235322 learned_addr learned my addr 172.21.15.97:0/670235322 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c51e4700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 -- 172.21.15.97:0/670235322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 msgr2=0x7f82c019b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c61e6700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c0101240 0x7f82c019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c019b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 -- 172.21.15.97:0/670235322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c0101240 msgr2=0x7f82c019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c0101240 0x7f82c019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 -- 172.21.15.97:0/670235322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8009d00 con 0x7f82c00ff6b0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c51e4700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c59e5700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82b000c970 tx=0x7f82b000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0004220 con 0x7f82c00ff6b0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82b0007d60 con 0x7f82c00ff6b0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c019e900 con 0x7f82c00ff6b0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0003d00 con 0x7f82c00ff6b0 2023-10-08T09:55:14.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.611+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c019ee10 con 0x7f82c00ff6b0 2023-10-08T09:55:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.612+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0106a60 con 0x7f82c00ff6b0 2023-10-08T09:55:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.613+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f82b0004380 con 0x7f82c00ff6b0 2023-10-08T09:55:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.614+0000 7f82b6ffd700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 0x7f82ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.614+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82b00ccf70 con 0x7f82c00ff6b0 2023-10-08T09:55:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.614+0000 7f82c51e4700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 0x7f82ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.614+0000 7f82c51e4700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 0x7f82ac0a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005fd0 tx=0x7f82bc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.617+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f82b00962c0 con 0x7f82c00ff6b0 2023-10-08T09:55:14.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.807+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82c010aed0 con 0x7f82c00ff6b0 2023-10-08T09:55:14.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.807+0000 7f82b6ffd700 1 -- 172.21.15.97:0/670235322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82b001d0a0 con 0x7f82c00ff6b0 2023-10-08T09:55:14.809 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.809+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 msgr2=0x7f82ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.809+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 0x7f82ac0a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005fd0 tx=0x7f82bc005dc0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.809+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 msgr2=0x7f82c0069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.809+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82b000c970 tx=0x7f82b000cd30 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 shutdown_connections 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f82ac0a1480 0x7f82ac0a3940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c0101240 0x7f82c019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c00ff6b0 0x7f82c0069550 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 --2- 172.21.15.97:0/670235322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82c01008a0 0x7f82c019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 >> 172.21.15.97:0/670235322 conn(0x7f82c00fac40 msgr2=0x7f82c0103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 shutdown_connections 2023-10-08T09:55:14.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.810+0000 7f82c7c49700 1 -- 172.21.15.97:0/670235322 wait complete. 2023-10-08T09:55:14.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:14.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.947+0000 7fa594e21700 1 -- 172.21.15.97:0/1201559856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5901007c0 msgr2=0x7fa590100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.947+0000 7fa594e21700 1 --2- 172.21.15.97:0/1201559856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5901007c0 0x7fa590100be0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5840099c0 tx=0x7fa584009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.947+0000 7fa594e21700 1 -- 172.21.15.97:0/1201559856 shutdown_connections 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.947+0000 7fa594e21700 1 --2- 172.21.15.97:0/1201559856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa590102350 0x7fa590108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.947+0000 7fa594e21700 1 --2- 172.21.15.97:0/1201559856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5901007c0 0x7fa590100be0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 --2- 172.21.15.97:0/1201559856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901019b0 0x7fa590101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- 172.21.15.97:0/1201559856 >> 172.21.15.97:0/1201559856 conn(0x7fa5900fbd30 msgr2=0x7fa5900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- 172.21.15.97:0/1201559856 shutdown_connections 2023-10-08T09:55:14.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- 172.21.15.97:0/1201559856 wait complete. 2023-10-08T09:55:14.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 Processor -- start 2023-10-08T09:55:14.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- start start 2023-10-08T09:55:14.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901007c0 0x7fa590196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5901019b0 0x7fa590196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019e240 con 0x7fa590102350 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019e380 con 0x7fa5901019b0 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.948+0000 7fa594e21700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59019e510 con 0x7fa5901007c0 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56152/0 (socket says 172.21.15.97:56152) 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 -- 172.21.15.97:0/3475245052 learned_addr learned my addr 172.21.15.97:0/3475245052 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58e59c700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901007c0 0x7fa590196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58dd9b700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5901019b0 0x7fa590196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 -- 172.21.15.97:0/3475245052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901007c0 msgr2=0x7fa590196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901007c0 0x7fa590196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 -- 172.21.15.97:0/3475245052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5901019b0 msgr2=0x7fa590196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5901019b0 0x7fa590196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 -- 172.21.15.97:0/3475245052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840095f0 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa58ed9d700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa58800ef30 tx=0x7fa58800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588018070 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59019e790 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58800f040 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800cdb0 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.949+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59019ec50 con 0x7fa590102350 2023-10-08T09:55:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.950+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa590107c90 con 0x7fa590102350 2023-10-08T09:55:14.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.952+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa588003a00 con 0x7fa590102350 2023-10-08T09:55:14.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.952+0000 7fa57f7fe700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 0x7fa5780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:14.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.952+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5880cd5f0 con 0x7fa590102350 2023-10-08T09:55:14.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.952+0000 7fa58e59c700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 0x7fa5780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:14.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.953+0000 7fa58e59c700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 0x7fa5780a3940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa584003730 tx=0x7fa584000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:14.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:14.955+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5880969d0 con 0x7fa590102350 2023-10-08T09:55:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.146+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa59010c0f0 con 0x7fa590102350 2023-10-08T09:55:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.147+0000 7fa57f7fe700 1 -- 172.21.15.97:0/3475245052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa588099ff0 con 0x7fa590102350 2023-10-08T09:55:15.148 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 msgr2=0x7fa5780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 0x7fa5780a3940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa584003730 tx=0x7fa584000bc0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 msgr2=0x7fa59019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa58800ef30 tx=0x7fa58800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 shutdown_connections 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa5780a1480 0x7fa5780a3940 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa590102350 0x7fa59019dc70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5901007c0 0x7fa590196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 --2- 172.21.15.97:0/3475245052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5901019b0 0x7fa590196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 >> 172.21.15.97:0/3475245052 conn(0x7fa5900fbd30 msgr2=0x7fa590104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 shutdown_connections 2023-10-08T09:55:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.149+0000 7fa594e21700 1 -- 172.21.15.97:0/3475245052 wait complete. 2023-10-08T09:55:15.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 -- 172.21.15.97:0/1909790799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f00ff5b0 msgr2=0x7f04f00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1909790799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f00ff5b0 0x7f04f00ff9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f04e40099c0 tx=0x7f04e4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 -- 172.21.15.97:0/1909790799 shutdown_connections 2023-10-08T09:55:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1909790799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f0101140 0x7f04f01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1909790799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f00ff5b0 0x7f04f00ff9d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1909790799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 0x7f04f0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 -- 172.21.15.97:0/1909790799 >> 172.21.15.97:0/1909790799 conn(0x7f04f00fab40 msgr2=0x7f04f00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 -- 172.21.15.97:0/1909790799 shutdown_connections 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.283+0000 7f04f48d6700 1 -- 172.21.15.97:0/1909790799 wait complete. 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 Processor -- start 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 -- start start 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f00ff5b0 0x7f04f00695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 0x7f04f019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f019e130 con 0x7f04f0101140 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f019e2a0 con 0x7f04f01007a0 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04f48d6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f019e460 con 0x7f04f00ff5b0 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56166/0 (socket says 172.21.15.97:56166) 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 -- 172.21.15.97:0/1533878188 learned_addr learned my addr 172.21.15.97:0/1533878188 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04edd9b700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 0x7f04f019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04ee59c700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f00ff5b0 0x7f04f00695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 -- 172.21.15.97:0/1533878188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f00ff5b0 msgr2=0x7f04f00695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f00ff5b0 0x7f04f00695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 -- 172.21.15.97:0/1533878188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 msgr2=0x7f04f019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.284+0000 7f04eed9d700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 0x7f04f019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04eed9d700 1 -- 172.21.15.97:0/1533878188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e40095f0 con 0x7f04f0101140 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04eed9d700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007ee0 tx=0x7f04e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8017070 con 0x7f04f0101140 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04e800f040 con 0x7f04f0101140 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f019e6e0 con 0x7f04f0101140 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8005580 con 0x7f04f0101140 2023-10-08T09:55:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.285+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f019ec40 con 0x7f04f0101140 2023-10-08T09:55:15.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.286+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f0106a80 con 0x7f04f0101140 2023-10-08T09:55:15.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.287+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f04e800db20 con 0x7f04f0101140 2023-10-08T09:55:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.287+0000 7f04df7fe700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 0x7f04d80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.288+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f04e80cc6a0 con 0x7f04f0101140 2023-10-08T09:55:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.288+0000 7f04ee59c700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 0x7f04d80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.288+0000 7f04ee59c700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 0x7f04d80a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003820 tx=0x7f04e4005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:15.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.292+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f04e80959f0 con 0x7f04f0101140 2023-10-08T09:55:15.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:15 smithi097 ceph-mon[99491]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:15.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/530875010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/670235322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3475245052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:15 smithi178 ceph-mon[103156]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/530875010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/670235322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3475245052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:15 smithi186 ceph-mon[104393]: pgmap v301: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/530875010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/670235322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3475245052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:15.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.533+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f0066bc0 con 0x7f04f0101140 2023-10-08T09:55:15.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.533+0000 7f04df7fe700 1 -- 172.21.15.97:0/1533878188 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f04e8099010 con 0x7f04f0101140 2023-10-08T09:55:15.534 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:15.534 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:15.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.535+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 msgr2=0x7f04d80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.535+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 0x7f04d80a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003820 tx=0x7f04e4005920 comp rx=0 tx=0).stop 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.535+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 msgr2=0x7f04f019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.535+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007ee0 tx=0x7f04e800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 shutdown_connections 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04d80a1500 0x7f04d80a39c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f0101140 0x7f04f019d940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04f00ff5b0 0x7f04f00695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 --2- 172.21.15.97:0/1533878188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f04f01007a0 0x7f04f019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 >> 172.21.15.97:0/1533878188 conn(0x7f04f00fab40 msgr2=0x7f04f0103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 shutdown_connections 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.536+0000 7f04f48d6700 1 -- 172.21.15.97:0/1533878188 wait complete. 2023-10-08T09:55:15.538 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:15.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 -- 172.21.15.97:0/2075424191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac0ff5a0 msgr2=0x7fc1ac0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/2075424191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac0ff9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00099c0 tx=0x7fc1a0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 -- 172.21.15.97:0/2075424191 shutdown_connections 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/2075424191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac101130 0x7fc1ac1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/2075424191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac0ff9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/2075424191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac100790 0x7fc1ac100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 -- 172.21.15.97:0/2075424191 >> 172.21.15.97:0/2075424191 conn(0x7fc1ac0fab10 msgr2=0x7fc1ac0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.668+0000 7fc1b2da3700 1 -- 172.21.15.97:0/2075424191 shutdown_connections 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 -- 172.21.15.97:0/2075424191 wait complete. 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 Processor -- start 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 -- start start 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 0x7fc1ac195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac19d080 con 0x7fc1ac101130 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac19d1c0 con 0x7fc1ac0ff5a0 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b2da3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac19d350 con 0x7fc1ac100790 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b1340700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b1340700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56180/0 (socket says 172.21.15.97:56180) 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b1340700 1 -- 172.21.15.97:0/44475538 learned_addr learned my addr 172.21.15.97:0/44475538 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1abfff700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 0x7fc1ac195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.669+0000 7fc1b0b3f700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 -- 172.21.15.97:0/44475538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 msgr2=0x7fc1ac195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 0x7fc1ac195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 -- 172.21.15.97:0/44475538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac0ff5a0 msgr2=0x7fc1ac195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac195550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 -- 172.21.15.97:0/44475538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00095f0 con 0x7fc1ac101130 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1abfff700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 0x7fc1ac195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b1340700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400ada0 tx=0x7fc1a400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400f070 con 0x7fc1ac101130 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1a4007b30 con 0x7fc1ac101130 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac19d630 con 0x7fc1ac101130 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4005440 con 0x7fc1ac101130 2023-10-08T09:55:15.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.670+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac19daf0 con 0x7fc1ac101130 2023-10-08T09:55:15.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.671+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac106a40 con 0x7fc1ac101130 2023-10-08T09:55:15.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.672+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc1a4004ad0 con 0x7fc1ac101130 2023-10-08T09:55:15.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.673+0000 7fc1a9ffb700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 0x7fc19c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.673+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1a40cc690 con 0x7fc1ac101130 2023-10-08T09:55:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.673+0000 7fc1b0b3f700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 0x7fc19c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.673+0000 7fc1b0b3f700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 0x7fc19c0a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b580 tx=0x7fc1a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:15.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.677+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc1a4095a70 con 0x7fc1ac101130 2023-10-08T09:55:15.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.869+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1ac067450 con 0x7fc1ac101130 2023-10-08T09:55:15.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.871+0000 7fc1a9ffb700 1 -- 172.21.15.97:0/44475538 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7fc1a4099090 con 0x7fc1ac101130 2023-10-08T09:55:15.872 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:15.872 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 msgr2=0x7fc19c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 0x7fc19c0a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b580 tx=0x7fc1a0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 msgr2=0x7fc1ac19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400ada0 tx=0x7fc1a400c330 comp rx=0 tx=0).stop 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 shutdown_connections 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fc19c0a1480 0x7fc19c0a3940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1ac101130 0x7fc1ac19cab0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1ac0ff5a0 0x7fc1ac195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 --2- 172.21.15.97:0/44475538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac100790 0x7fc1ac195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:15.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 >> 172.21.15.97:0/44475538 conn(0x7fc1ac0fab10 msgr2=0x7fc1ac103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:15.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 shutdown_connections 2023-10-08T09:55:15.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:15.873+0000 7fc1b2da3700 1 -- 172.21.15.97:0/44475538 wait complete. 2023-10-08T09:55:15.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:15.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 -- 172.21.15.97:0/1210228976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb281012c0 msgr2=0x7feb28107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 --2- 172.21.15.97:0/1210228976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb281012c0 0x7feb28107b50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feb2000b0a0 tx=0x7feb2000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 -- 172.21.15.97:0/1210228976 shutdown_connections 2023-10-08T09:55:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 --2- 172.21.15.97:0/1210228976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb281012c0 0x7feb28107b50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 --2- 172.21.15.97:0/1210228976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb280ffbf0 0x7feb28100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 --2- 172.21.15.97:0/1210228976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb28100900 0x7feb28100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 -- 172.21.15.97:0/1210228976 >> 172.21.15.97:0/1210228976 conn(0x7feb280fa3c0 msgr2=0x7feb280fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 -- 172.21.15.97:0/1210228976 shutdown_connections 2023-10-08T09:55:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.011+0000 7feb2d462700 1 -- 172.21.15.97:0/1210228976 wait complete. 2023-10-08T09:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 Processor -- start 2023-10-08T09:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 -- start start 2023-10-08T09:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 0x7feb2819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 0x7feb2819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2819e350 con 0x7feb280ffbf0 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2819e4c0 con 0x7feb28100900 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb2d462700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2819e680 con 0x7feb281012c0 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb26ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb26ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56190/0 (socket says 172.21.15.97:56190) 2023-10-08T09:55:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.012+0000 7feb26ffd700 1 -- 172.21.15.97:0/3831522972 learned_addr learned my addr 172.21.15.97:0/3831522972 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb267fc700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 0x7feb2819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb277fe700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 0x7feb2819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 -- 172.21.15.97:0/3831522972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 msgr2=0x7feb2819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 0x7feb2819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 -- 172.21.15.97:0/3831522972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 msgr2=0x7feb2819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 0x7feb2819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 -- 172.21.15.97:0/3831522972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009d00 con 0x7feb280ffbf0 2023-10-08T09:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb267fc700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 0x7feb2819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb277fe700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 0x7feb2819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:16.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb26ffd700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e970 tx=0x7feb1000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10009790 con 0x7feb280ffbf0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2819e900 con 0x7feb280ffbf0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb100044e0 con 0x7feb280ffbf0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2819ee60 con 0x7feb280ffbf0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.013+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10003e30 con 0x7feb280ffbf0 2023-10-08T09:55:16.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.014+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb28106b40 con 0x7feb280ffbf0 2023-10-08T09:55:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.015+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feb10004000 con 0x7feb280ffbf0 2023-10-08T09:55:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.015+0000 7feb1ffff700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 0x7feb140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.016+0000 7feb267fc700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 0x7feb140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.016+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb10014070 con 0x7feb280ffbf0 2023-10-08T09:55:16.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.016+0000 7feb267fc700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 0x7feb140a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb280ff820 tx=0x7feb18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.020+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feb10096680 con 0x7feb280ffbf0 2023-10-08T09:55:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.211+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb2810b110 con 0x7feb280ffbf0 2023-10-08T09:55:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.211+0000 7feb1ffff700 1 -- 172.21.15.97:0/3831522972 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb10099ca0 con 0x7feb280ffbf0 2023-10-08T09:55:16.212 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 msgr2=0x7feb140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 0x7feb140a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb280ff820 tx=0x7feb18009260 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 msgr2=0x7feb280ff280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e970 tx=0x7feb1000ed30 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 shutdown_connections 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feb140a1480 0x7feb140a3940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb281012c0 0x7feb2819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb280ffbf0 0x7feb280ff280 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 --2- 172.21.15.97:0/3831522972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb28100900 0x7feb2819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 >> 172.21.15.97:0/3831522972 conn(0x7feb280fa3c0 msgr2=0x7feb28103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 shutdown_connections 2023-10-08T09:55:16.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.214+0000 7feb2d462700 1 -- 172.21.15.97:0/3831522972 wait complete. 2023-10-08T09:55:16.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:16.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 -- 172.21.15.97:0/2901954273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c0ff6a0 msgr2=0x7f441c0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 --2- 172.21.15.97:0/2901954273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c0ff6a0 0x7f441c0ffac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f440c0099c0 tx=0x7f440c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 -- 172.21.15.97:0/2901954273 shutdown_connections 2023-10-08T09:55:16.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 --2- 172.21.15.97:0/2901954273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c101230 0x7f441c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 --2- 172.21.15.97:0/2901954273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c0ff6a0 0x7f441c0ffac0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 --2- 172.21.15.97:0/2901954273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c100890 0x7f441c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 -- 172.21.15.97:0/2901954273 >> 172.21.15.97:0/2901954273 conn(0x7f441c0fac10 msgr2=0x7f441c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 -- 172.21.15.97:0/2901954273 shutdown_connections 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.350+0000 7f4421e6e700 1 -- 172.21.15.97:0/2901954273 wait complete. 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 Processor -- start 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 -- start start 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 0x7f441c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c100890 0x7f441c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c19e210 con 0x7f441c101230 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c19e350 con 0x7f441c100890 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.351+0000 7f4421e6e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c19e4e0 con 0x7f441c0ff6a0 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56208/0 (socket says 172.21.15.97:56208) 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 -- 172.21.15.97:0/1847935533 learned_addr learned my addr 172.21.15.97:0/1847935533 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441affd700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c100890 0x7f441c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441b7fe700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 0x7f441c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 -- 172.21.15.97:0/1847935533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 msgr2=0x7f441c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 0x7f441c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 -- 172.21.15.97:0/1847935533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c100890 msgr2=0x7f441c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c100890 0x7f441c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 -- 172.21.15.97:0/1847935533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0095f0 con 0x7f441c101230 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441b7fe700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 0x7f441c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f441bfff700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4414007ee0 tx=0x7f441400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414017070 con 0x7f441c101230 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c19e7c0 con 0x7f441c101230 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441400f040 con 0x7f441c101230 2023-10-08T09:55:16.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414005740 con 0x7f441c101230 2023-10-08T09:55:16.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.352+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c19ec80 con 0x7f441c101230 2023-10-08T09:55:16.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.353+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c067530 con 0x7f441c101230 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.354+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f441400dc60 con 0x7f441c101230 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.355+0000 7f4418ff9700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 0x7f44040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.355+0000 7f441b7fe700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 0x7f44040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.355+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f44140cc8a0 con 0x7f441c101230 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.355+0000 7f441b7fe700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 0x7f44040a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b580 tx=0x7f440c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.359+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4414095b90 con 0x7f441c101230 2023-10-08T09:55:16.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:16 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1533878188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:16 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/44475538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:16 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1533878188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:16 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/44475538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:16 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1533878188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:16 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/44475538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:16.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.552+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f441c19ef40 con 0x7f441c101230 2023-10-08T09:55:16.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.553+0000 7f4418ff9700 1 -- 172.21.15.97:0/1847935533 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44140991b0 con 0x7f441c101230 2023-10-08T09:55:16.554 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 msgr2=0x7f44040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 0x7f44040a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b580 tx=0x7f440c005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 msgr2=0x7f441c19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4414007ee0 tx=0x7f441400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 shutdown_connections 2023-10-08T09:55:16.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f44040a1480 0x7f44040a3940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f441c101230 0x7f441c19dc40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f441c0ff6a0 0x7f441c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 --2- 172.21.15.97:0/1847935533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f441c100890 0x7f441c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 >> 172.21.15.97:0/1847935533 conn(0x7f441c0fac10 msgr2=0x7f441c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 shutdown_connections 2023-10-08T09:55:16.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.555+0000 7f4421e6e700 1 -- 172.21.15.97:0/1847935533 wait complete. 2023-10-08T09:55:16.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.691+0000 7f40ce02d700 1 -- 172.21.15.97:0/1881620446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 msgr2=0x7f40c81036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.691+0000 7f40ce02d700 1 --2- 172.21.15.97:0/1881620446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c81036a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009990 tx=0x7f40b8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 -- 172.21.15.97:0/1881620446 shutdown_connections 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 --2- 172.21.15.97:0/1881620446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8106510 0x7f40c8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 --2- 172.21.15.97:0/1881620446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c81036a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 --2- 172.21.15.97:0/1881620446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8103be0 0x7f40c8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 -- 172.21.15.97:0/1881620446 >> 172.21.15.97:0/1881620446 conn(0x7f40c80fac40 msgr2=0x7f40c80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 -- 172.21.15.97:0/1881620446 shutdown_connections 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 -- 172.21.15.97:0/1881620446 wait complete. 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.692+0000 7f40ce02d700 1 Processor -- start 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 -- start start 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8103be0 0x7f40c8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8106510 0x7f40c819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819e240 con 0x7f40c81012b0 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819e380 con 0x7f40c8103be0 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819e510 con 0x7f40c8106510 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56216/0 (socket says 172.21.15.97:56216) 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 -- 172.21.15.97:0/309888504 learned_addr learned my addr 172.21.15.97:0/309888504 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c6ffd700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8103be0 0x7f40c8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c7fff700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8106510 0x7f40c819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 -- 172.21.15.97:0/309888504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8106510 msgr2=0x7f40c819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8106510 0x7f40c819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 -- 172.21.15.97:0/309888504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8103be0 msgr2=0x7f40c8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8103be0 0x7f40c8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 -- 172.21.15.97:0/309888504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80095f0 con 0x7f40c81012b0 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c77fe700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f40b80095c0 tx=0x7f40b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b801d070 con 0x7f40c81012b0 2023-10-08T09:55:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c819e790 con 0x7f40c81012b0 2023-10-08T09:55:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b8022470 con 0x7f40c81012b0 2023-10-08T09:55:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.693+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8017650 con 0x7f40c81012b0 2023-10-08T09:55:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.694+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c819ebc0 con 0x7f40c81012b0 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.695+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c80fc7c0 con 0x7f40c81012b0 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.696+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f40b8022850 con 0x7f40c81012b0 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.696+0000 7f40c4ff9700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 0x7f40b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.696+0000 7f40c6ffd700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 0x7f40b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.696+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f40b80ce8f0 con 0x7f40c81012b0 2023-10-08T09:55:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.697+0000 7f40c6ffd700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 0x7f40b00a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005950 tx=0x7f40bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:16.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.700+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f40b8097df0 con 0x7f40c81012b0 2023-10-08T09:55:16.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.890+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40c810c0d0 con 0x7f40c81012b0 2023-10-08T09:55:16.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.891+0000 7f40c4ff9700 1 -- 172.21.15.97:0/309888504 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40b8027020 con 0x7f40c81012b0 2023-10-08T09:55:16.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:16.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 msgr2=0x7f40b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 0x7f40b00a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005950 tx=0x7f40bc00b410 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 msgr2=0x7f40c8196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f40b80095c0 tx=0x7f40b800f690 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 shutdown_connections 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f40b00a1480 0x7f40b00a3940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40c8106510 0x7f40c819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c81012b0 0x7f40c8196710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 --2- 172.21.15.97:0/309888504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40c8103be0 0x7f40c8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 >> 172.21.15.97:0/309888504 conn(0x7f40c80fac40 msgr2=0x7f40c8103140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 shutdown_connections 2023-10-08T09:55:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:16.893+0000 7f40ce02d700 1 -- 172.21.15.97:0/309888504 wait complete. 2023-10-08T09:55:16.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 -- 172.21.15.97:0/3872881012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a4100890 msgr2=0x7ff2a4100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/3872881012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a4100890 0x7ff2a4100cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00099c0 tx=0x7ff2a0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 -- 172.21.15.97:0/3872881012 shutdown_connections 2023-10-08T09:55:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/3872881012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4101230 0x7ff2a4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/3872881012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a40ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/3872881012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a4100890 0x7ff2a4100cf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 -- 172.21.15.97:0/3872881012 >> 172.21.15.97:0/3872881012 conn(0x7ff2a40fac10 msgr2=0x7ff2a40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 -- 172.21.15.97:0/3872881012 shutdown_connections 2023-10-08T09:55:17.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.030+0000 7ff2ac22c700 1 -- 172.21.15.97:0/3872881012 wait complete. 2023-10-08T09:55:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 Processor -- start 2023-10-08T09:55:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 -- start start 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 0x7ff2a4195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 0x7ff2a419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d120 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d260 con 0x7ff2a4100890 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2ac22c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d3f0 con 0x7ff2a4101230 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2a9fc8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2a9fc8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56230/0 (socket says 172.21.15.97:56230) 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2a9fc8700 1 -- 172.21.15.97:0/1253331164 learned_addr learned my addr 172.21.15.97:0/1253331164 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:17.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2a97c7700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 0x7ff2a4195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.031+0000 7ff2aa7c9700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 0x7ff2a419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 -- 172.21.15.97:0/1253331164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 msgr2=0x7ff2a419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 0x7ff2a419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 -- 172.21.15.97:0/1253331164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 msgr2=0x7ff2a4195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 0x7ff2a4195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 -- 172.21.15.97:0/1253331164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00095f0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2aa7c9700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 0x7ff2a419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a97c7700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 0x7ff2a4195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2a9fc8700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff29400e970 tx=0x7ff29400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294009790 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29400eec0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a419d6d0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294003cf0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.032+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a419dbe0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.033+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4106a50 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.034+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff29400ccd0 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.034+0000 7ff29affd700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 0x7ff2900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.035+0000 7ff2a97c7700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 0x7ff2900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.035+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff294014070 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.035+0000 7ff2a97c7700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 0x7ff2900a3940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0003730 tx=0x7ff2a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.038+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff294096760 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.174 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:17 smithi097 ceph-mon[99491]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:17.175 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3831522972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.229+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2a410af10 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.229+0000 7ff29affd700 1 -- 172.21.15.97:0/1253331164 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff294099d80 con 0x7ff2a40ff6a0 2023-10-08T09:55:17.230 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 msgr2=0x7ff2900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 0x7ff2900a3940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0003730 tx=0x7ff2a001a040 comp rx=0 tx=0).stop 2023-10-08T09:55:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 msgr2=0x7ff2a41955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff29400e970 tx=0x7ff29400ec80 comp rx=0 tx=0).stop 2023-10-08T09:55:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 shutdown_connections 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff2900a1480 0x7ff2900a3940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4101230 0x7ff2a419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2a40ff6a0 0x7ff2a41955f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 --2- 172.21.15.97:0/1253331164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4100890 0x7ff2a4195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 >> 172.21.15.97:0/1253331164 conn(0x7ff2a40fac10 msgr2=0x7ff2a4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 shutdown_connections 2023-10-08T09:55:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.231+0000 7ff2ac22c700 1 -- 172.21.15.97:0/1253331164 wait complete. 2023-10-08T09:55:17.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- 172.21.15.97:0/756781432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 msgr2=0x7fb968100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 --2- 172.21.15.97:0/756781432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968100c60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb964009990 tx=0x7fb964009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- 172.21.15.97:0/756781432 shutdown_connections 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 --2- 172.21.15.97:0/756781432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9681011a0 0x7fb968107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 --2- 172.21.15.97:0/756781432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9680ffad0 0x7fb9680ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 --2- 172.21.15.97:0/756781432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968100c60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- 172.21.15.97:0/756781432 >> 172.21.15.97:0/756781432 conn(0x7fb9680fa2c0 msgr2=0x7fb9680fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- 172.21.15.97:0/756781432 shutdown_connections 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- 172.21.15.97:0/756781432 wait complete. 2023-10-08T09:55:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 Processor -- start 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.365+0000 7fb970079700 1 -- start start 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 0x7fb968103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 0x7fb968102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9681028a0 con 0x7fb9681007e0 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968102a10 con 0x7fb9681011a0 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9681a06c0 con 0x7fb9680ffad0 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56246/0 (socket says 172.21.15.97:56246) 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 -- 172.21.15.97:0/909456740 learned_addr learned my addr 172.21.15.97:0/909456740 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96de15700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 0x7fb968103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96e616700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 0x7fb968102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 -- 172.21.15.97:0/909456740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 msgr2=0x7fb968103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 0x7fb968103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 -- 172.21.15.97:0/909456740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 msgr2=0x7fb968102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 0x7fb968102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 -- 172.21.15.97:0/909456740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640095f0 con 0x7fb9681007e0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96de15700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 0x7fb968103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96e616700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 0x7fb968102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb96d614700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb964004730 tx=0x7fb964009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96401d070 con 0x7fb9681007e0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9681a08f0 con 0x7fb9681007e0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9640040d0 con 0x7fb9681007e0 2023-10-08T09:55:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964005290 con 0x7fb9681007e0 2023-10-08T09:55:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.366+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9681a0d10 con 0x7fb9681007e0 2023-10-08T09:55:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.367+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb968066bc0 con 0x7fb9681007e0 2023-10-08T09:55:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.368+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb9640048a0 con 0x7fb9681007e0 2023-10-08T09:55:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.369+0000 7fb95effd700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 0x7fb9540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.369+0000 7fb96de15700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 0x7fb9540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.369+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9640ce3f0 con 0x7fb9681007e0 2023-10-08T09:55:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.369+0000 7fb96de15700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 0x7fb9540a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9680673a0 tx=0x7fb958005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.372+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb9640977d0 con 0x7fb9681007e0 2023-10-08T09:55:17.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1847935533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/309888504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:17 smithi178 ceph-mon[103156]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3831522972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1847935533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/309888504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:17 smithi186 ceph-mon[104393]: pgmap v302: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3831522972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1847935533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/309888504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.606+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb96810adf0 con 0x7fb9681007e0 2023-10-08T09:55:17.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.606+0000 7fb95effd700 1 -- 172.21.15.97:0/909456740 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7fb964026090 con 0x7fb9681007e0 2023-10-08T09:55:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:17.607 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 msgr2=0x7fb9540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 0x7fb9540a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9680673a0 tx=0x7fb958005f90 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 msgr2=0x7fb968101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb964004730 tx=0x7fb964009580 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 shutdown_connections 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9540a1480 0x7fb9540a3940 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9681011a0 0x7fb968102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9680ffad0 0x7fb968103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 --2- 172.21.15.97:0/909456740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9681007e0 0x7fb968101bc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 >> 172.21.15.97:0/909456740 conn(0x7fb9680fa2c0 msgr2=0x7fb9680fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 shutdown_connections 2023-10-08T09:55:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.609+0000 7fb970079700 1 -- 172.21.15.97:0/909456740 wait complete. 2023-10-08T09:55:17.611 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:17.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:17.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- 172.21.15.97:0/3194293946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba01007c0 msgr2=0x7f9ba0100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/3194293946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba01007c0 0x7f9ba0100c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0099c0 tx=0x7f9b9c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- 172.21.15.97:0/3194293946 shutdown_connections 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/3194293946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba0101160 0x7f9ba0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/3194293946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba00ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/3194293946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba01007c0 0x7f9ba0100c20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- 172.21.15.97:0/3194293946 >> 172.21.15.97:0/3194293946 conn(0x7f9ba00fab40 msgr2=0x7f9ba00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- 172.21.15.97:0/3194293946 shutdown_connections 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- 172.21.15.97:0/3194293946 wait complete. 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 Processor -- start 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.742+0000 7f9ba85fe700 1 -- start start 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba01007c0 0x7f9ba0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba0101160 0x7f9ba019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01a0350 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01a0490 con 0x7f9ba01007c0 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01a0620 con 0x7f9ba0101160 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56258/0 (socket says 172.21.15.97:56258) 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 -- 172.21.15.97:0/1592924833 learned_addr learned my addr 172.21.15.97:0/1592924833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 -- 172.21.15.97:0/1592924833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba0101160 msgr2=0x7f9ba019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba0101160 0x7f9ba019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 -- 172.21.15.97:0/1592924833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba01007c0 msgr2=0x7f9ba0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba01007c0 0x7f9ba0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 -- 172.21.15.97:0/1592924833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0095f0 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba639a700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d980 tx=0x7f9b9000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000bb60 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba01a0900 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b90007620 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000f620 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.743+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba01a0e60 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.744+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0066bc0 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.745+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9b90010460 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.746+0000 7f9b977fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 0x7f9b8c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.746+0000 7f9ba5b99700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 0x7f9b8c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.746+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b900cd5c0 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.746+0000 7f9ba5b99700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 0x7f9b8c0a39c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0036f0 tx=0x7f9b9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:17.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.749+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9b900968b0 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.941+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ba010adf0 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.943+0000 7f9b977fe700 1 -- 172.21.15.97:0/1592924833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7f9b9000de60 con 0x7f9ba00ff5d0 2023-10-08T09:55:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:17.944 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 msgr2=0x7f9b8c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 0x7f9b8c0a39c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0036f0 tx=0x7f9b9c000bc0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 msgr2=0x7f9ba0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d980 tx=0x7f9b9000dd40 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 shutdown_connections 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b8c0a1500 0x7f9b8c0a39c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ba0101160 0x7f9ba019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba00ff5d0 0x7f9ba0198820 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 --2- 172.21.15.97:0/1592924833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba01007c0 0x7f9ba0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 >> 172.21.15.97:0/1592924833 conn(0x7f9ba00fab40 msgr2=0x7f9ba0068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:17.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 shutdown_connections 2023-10-08T09:55:17.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:17.945+0000 7f9ba85fe700 1 -- 172.21.15.97:0/1592924833 wait complete. 2023-10-08T09:55:17.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:18.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 -- 172.21.15.97:0/1637535727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678100bb0 msgr2=0x7ff678100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/1637535727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678100bb0 0x7ff678100fd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff668009990 tx=0x7ff668009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 -- 172.21.15.97:0/1637535727 shutdown_connections 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/1637535727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6781001f0 0x7ff678107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/1637535727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678100bb0 0x7ff678100fd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/1637535727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6780ff850 0x7ff6780ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 -- 172.21.15.97:0/1637535727 >> 172.21.15.97:0/1637535727 conn(0x7ff6780fa2c0 msgr2=0x7ff6780fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 -- 172.21.15.97:0/1637535727 shutdown_connections 2023-10-08T09:55:18.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.078+0000 7ff67d2c0700 1 -- 172.21.15.97:0/1637535727 wait complete. 2023-10-08T09:55:18.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 Processor -- start 2023-10-08T09:55:18.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 -- start start 2023-10-08T09:55:18.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 0x7ff678195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 0x7ff67819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67819d080 con 0x7ff6780ff850 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67819d1c0 con 0x7ff6781001f0 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff67d2c0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67819d350 con 0x7ff678100bb0 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff676ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff676ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56272/0 (socket says 172.21.15.97:56272) 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.079+0000 7ff676ffd700 1 -- 172.21.15.97:0/154792861 learned_addr learned my addr 172.21.15.97:0/154792861 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff6767fc700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 0x7ff678195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff6777fe700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 0x7ff67819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 -- 172.21.15.97:0/154792861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 msgr2=0x7ff67819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 0x7ff67819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 -- 172.21.15.97:0/154792861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 msgr2=0x7ff678195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 0x7ff678195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 -- 172.21.15.97:0/154792861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680095f0 con 0x7ff6780ff850 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff6777fe700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 0x7ff67819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff6767fc700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 0x7ff678195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff676ffd700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b680 tx=0x7ff668005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66801d070 con 0x7ff6780ff850 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff66800fc80 con 0x7ff6780ff850 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67819d5d0 con 0x7ff6780ff850 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668017650 con 0x7ff6780ff850 2023-10-08T09:55:18.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.080+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67819d9f0 con 0x7ff6780ff850 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.081+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff678106aa0 con 0x7ff6780ff850 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.082+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff668021410 con 0x7ff6780ff850 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.082+0000 7ff65ffff700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 0x7ff6600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.083+0000 7ff6767fc700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 0x7ff6600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.083+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6680ce530 con 0x7ff6780ff850 2023-10-08T09:55:18.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.083+0000 7ff6767fc700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 0x7ff6600a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff66c005950 tx=0x7ff66c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.087+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6680979a0 con 0x7ff6780ff850 2023-10-08T09:55:18.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.323+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6780678a0 con 0x7ff6780ff850 2023-10-08T09:55:18.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.323+0000 7ff65ffff700 1 -- 172.21.15.97:0/154792861 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7ff66809afc0 con 0x7ff6780ff850 2023-10-08T09:55:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:18.325 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 msgr2=0x7ff6600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 0x7ff6600a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff66c005950 tx=0x7ff66c005f50 comp rx=0 tx=0).stop 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 msgr2=0x7ff678195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b680 tx=0x7ff668005e80 comp rx=0 tx=0).stop 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 shutdown_connections 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff6600a1480 0x7ff6600a3940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff678100bb0 0x7ff67819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6780ff850 0x7ff678195550 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 --2- 172.21.15.97:0/154792861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff6781001f0 0x7ff678195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.325+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 >> 172.21.15.97:0/154792861 conn(0x7ff6780fa2c0 msgr2=0x7ff678103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.326+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 shutdown_connections 2023-10-08T09:55:18.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.326+0000 7ff67d2c0700 1 -- 172.21.15.97:0/154792861 wait complete. 2023-10-08T09:55:18.328 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:18.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1253331164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:18.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/909456740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1592924833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T09:53:56.599415+0000', 'modified': '2023-10-08T09:54:24.185261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24562, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2717102390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2717102390}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2717102390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/3587378302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3587378302}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3587378302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/2015632800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2015632800}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2015632800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.186:6837/4190406762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 4190406762}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 4190406762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.186:6839/2120449823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2120449823}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2120449823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6839/1665352653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 1665352653}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 1665352653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T09:55:18.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-08T09:55:18.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T09:55:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1253331164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/909456740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1592924833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 -- 172.21.15.97:0/447506593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac101230 msgr2=0x7fecac107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 --2- 172.21.15.97:0/447506593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac101230 0x7fecac107ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feca400b0d0 tx=0x7feca400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 -- 172.21.15.97:0/447506593 shutdown_connections 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 --2- 172.21.15.97:0/447506593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac101230 0x7fecac107ae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 --2- 172.21.15.97:0/447506593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac0ff6a0 0x7fecac0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 --2- 172.21.15.97:0/447506593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 0x7fecac100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 -- 172.21.15.97:0/447506593 >> 172.21.15.97:0/447506593 conn(0x7fecac0fac10 msgr2=0x7fecac0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 -- 172.21.15.97:0/447506593 shutdown_connections 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.460+0000 7fecb1faf700 1 -- 172.21.15.97:0/447506593 wait complete. 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 Processor -- start 2023-10-08T09:55:18.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 -- start start 2023-10-08T09:55:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 0x7fecac19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 0x7fecac19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e330 con 0x7fecac0ff6a0 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e4a0 con 0x7fecac100890 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecb1faf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e660 con 0x7fecac101230 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56286/0 (socket says 172.21.15.97:56286) 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.461+0000 7fecab7fe700 1 -- 172.21.15.97:0/3505898652 learned_addr learned my addr 172.21.15.97:0/3505898652 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecaaffd700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 0x7fecac19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecabfff700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 0x7fecac19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 -- 172.21.15.97:0/3505898652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 msgr2=0x7fecac19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 0x7fecac19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 -- 172.21.15.97:0/3505898652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 msgr2=0x7fecac19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 0x7fecac19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 -- 172.21.15.97:0/3505898652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca4009d00 con 0x7fecac0ff6a0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecabfff700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 0x7fecac19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecab7fe700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00d980 tx=0x7fec9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7feca8ff9700 1 -- 172.21.15.97:0/3505898652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c004020 con 0x7fecac0ff6a0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac19e8e0 con 0x7fecac0ff6a0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7feca8ff9700 1 -- 172.21.15.97:0/3505898652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec9c009730 con 0x7fecac0ff6a0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7feca8ff9700 1 -- 172.21.15.97:0/3505898652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c003a30 con 0x7fecac0ff6a0 2023-10-08T09:55:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.462+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac19edf0 con 0x7fecac0ff6a0 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.464+0000 7feca8ff9700 1 -- 172.21.15.97:0/3505898652 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fec9c0098a0 con 0x7fecac0ff6a0 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.465+0000 7feca8ff9700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 0x7fec940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.465+0000 7fecaaffd700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 0x7fec940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.465+0000 7feca8ff9700 1 -- 172.21.15.97:0/3505898652 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec9c0ccd40 con 0x7fecac0ff6a0 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.465+0000 7fecaaffd700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 0x7fec940a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fecac068910 tx=0x7feca0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 Processor -- start 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 -- start start 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 0x56461a1cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a12e410 con 0x56461a1ca420 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a11fbd0 con 0x56461a1cad80 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecbe198b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a11d4f0 con 0x56461a1cbbc0 2023-10-08T09:55:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56294/0 (socket says 172.21.15.97:56294) 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 -- 172.21.15.97:0/4073097596 learned_addr learned my addr 172.21.15.97:0/4073097596 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecaaffd700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 0x56461a1cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 -- 172.21.15.97:0/4073097596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 msgr2=0x56461a1cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecabfff700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 -- 172.21.15.97:0/4073097596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 msgr2=0x56461a1cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 0x56461a1cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecab7fe700 1 -- 172.21.15.97:0/4073097596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564619f43d90 con 0x56461a1ca420 2023-10-08T09:55:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.466+0000 7fecaaffd700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 0x56461a1cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecab7fe700 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fec9c099c80 tx=0x7fec9c0996d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45f8b79e8013a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fec91ffb700 1 -- 172.21.15.97:0/4073097596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c09de40 con 0x56461a1ca420 2023-10-08T09:55:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fec91ffb700 1 -- 172.21.15.97:0/4073097596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec9c0a2080 con 0x56461a1ca420 2023-10-08T09:55:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fec91ffb700 1 -- 172.21.15.97:0/4073097596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0991e0 con 0x56461a1ca420 2023-10-08T09:55:18.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 -- 172.21.15.97:0/4073097596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 msgr2=0x56461a1ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fec9c099c80 tx=0x7fec9c0996d0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 -- 172.21.15.97:0/4073097596 shutdown_connections 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1ca420 0x56461a1ca840 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 --2- 172.21.15.97:0/4073097596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1cad80 0x56461a1cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 -- 172.21.15.97:0/4073097596 >> 172.21.15.97:0/4073097596 conn(0x56461a13a210 msgr2=0x56461a1c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 -- 172.21.15.97:0/4073097596 shutdown_connections 2023-10-08T09:55:18.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.467+0000 7fecbe198b80 1 -- 172.21.15.97:0/4073097596 wait complete. 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 -- start start 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 Processor -- start 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 0x56461a1d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a118de0 con 0x56461a1cad80 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a0b56c0 con 0x56461a1ca420 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a0b5360 con 0x56461a1cbbc0 2023-10-08T09:55:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecbe198b80 10 client.0 _ll_register_callbacks cb 0x564619f127b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56302/0 (socket says 172.21.15.97:56302) 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecaaffd700 1 -- 172.21.15.97:0/2866834825 learned_addr learned my addr 172.21.15.97:0/2866834825 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.468+0000 7fecabfff700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 0x56461a1d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 -- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 msgr2=0x56461a1db7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1db7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 -- 172.21.15.97:0/2866834825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 msgr2=0x56461a1d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 0x56461a1d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 -- 172.21.15.97:0/2866834825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac19edf0 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 0x56461a1d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feca001d790 tx=0x7feca00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac44a1b1ea979159 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0004230 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feca0004390 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56461a0e0e60 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feca0004550 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecbe198b80 10 client.15321 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564619f2cc10 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56461a0e0e60 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feca001bac0 con 0x56461a1cad80 2023-10-08T09:55:18.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.469+0000 7fecbe198b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecbe198b80 10 client.15321 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecbe198b80 20 client.15321 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a1dfe60 0x56461a1de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecbe198b80 4 client.15321 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56461a111dc0 con 0x56461a1dfe60 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a1dfe60 0x56461a1de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fec8affd700 10 client.15321 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.470+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a1dfe60 0x56461a1de910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b0ef041d7448cab server_cookie=2953ba49ca8af5a in_seq=0 out_seq=0 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fec8affd700 1 -- 172.21.15.97:0/2866834825 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56461a111dc0 con 0x56461a1dfe60 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fec8affd700 10 client.15321 handle_command_reply: tid=0 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 client.15321 shutdown 2023-10-08T09:55:18.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 msgr2=0x56461a1d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feca001d790 tx=0x7feca00047c0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 shutdown_connections 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a1dfe60 0x56461a1de910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1cbbc0 0x56461a1db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fec8affd700 0 client.15321 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1ca420 0x56461a1d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2866834825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cad80 0x56461a1d4760 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 >> 172.21.15.97:0/2866834825 conn(0x56461a13a210 msgr2=0x56461a1c0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 shutdown_connections 2023-10-08T09:55:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.471+0000 7fecbe198b80 1 -- 172.21.15.97:0/2866834825 wait complete. 2023-10-08T09:55:18.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.472+0000 7fecbe198b80 20 client.15321 trim_cache size 0 max 16384 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 Processor -- start 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 -- start start 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a1d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564619f998d0 con 0x56461a1cbbc0 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a111dc0 con 0x56461a1d21c0 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecbe198b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a0e0e60 con 0x56461a1ca420 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56310/0 (socket says 172.21.15.97:56310) 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecaaffd700 1 -- 172.21.15.97:0/1243874329 learned_addr learned my addr 172.21.15.97:0/1243874329 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecabfff700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a1d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.489+0000 7fecab7fe700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 -- 172.21.15.97:0/1243874329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 msgr2=0x56461a1d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 -- 172.21.15.97:0/1243874329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 msgr2=0x56461a1d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a1d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 -- 172.21.15.97:0/1243874329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564619f6a200 con 0x56461a1cbbc0 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecabfff700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a1d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecab7fe700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecaaffd700 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feca001bdf0 tx=0x7feca0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4f8f32c349f34b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fec927fc700 1 -- 172.21.15.97:0/1243874329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca001bc40 con 0x56461a1cbbc0 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fec927fc700 1 -- 172.21.15.97:0/1243874329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca001b810 con 0x56461a1cbbc0 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fec927fc700 1 -- 172.21.15.97:0/1243874329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0003d00 con 0x56461a1cbbc0 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 -- 172.21.15.97:0/1243874329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 msgr2=0x56461a1db760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feca001bdf0 tx=0x7feca0005ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 -- 172.21.15.97:0/1243874329 shutdown_connections 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a1d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 --2- 172.21.15.97:0/1243874329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1db760 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 -- 172.21.15.97:0/1243874329 >> 172.21.15.97:0/1243874329 conn(0x56461a13a210 msgr2=0x56461a1bcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.490+0000 7fecbe198b80 1 -- 172.21.15.97:0/1243874329 shutdown_connections 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 -- 172.21.15.97:0/1243874329 wait complete. 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 -- start start 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 Processor -- start 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a233850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a233d90 con 0x56461a1cbbc0 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a233ed0 con 0x56461a1d21c0 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56461a234040 con 0x56461a1ca420 2023-10-08T09:55:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecbe198b80 10 client.0 _ll_register_callbacks cb 0x7feca0011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56316/0 (socket says 172.21.15.97:56316) 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecaaffd700 1 -- 172.21.15.97:0/2901130103 learned_addr learned my addr 172.21.15.97:0/2901130103 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.491+0000 7fecabfff700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a233850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 -- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 msgr2=0x56461a1d86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 -- 172.21.15.97:0/2901130103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 msgr2=0x56461a233850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a233850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 -- 172.21.15.97:0/2901130103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564619f2cc10 con 0x56461a1cbbc0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecaaffd700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feca0009b20 tx=0x7feca0005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2252f07edd0ae692 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca001bc40 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feca00276a0 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec9c0a2080 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feca00056b0 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 10 client.15327 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564619f42260 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec9c0a2080 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feca003e020 con 0x56461a1cbbc0 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 10 client.15327 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 20 client.15327 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a238fe0 0x56461a235480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 4 client.15327 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.492+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56461a111ae0 con 0x56461a238fe0 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a238fe0 0x56461a235480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fec8affd700 10 client.15327 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecab7fe700 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a238fe0 0x56461a235480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e26a428752c60a67 server_cookie=f9790c698e5b6507 in_seq=0 out_seq=0 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fec8affd700 1 -- 172.21.15.97:0/2901130103 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56461a111ae0 con 0x56461a238fe0 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fec8affd700 10 client.15327 handle_command_reply: tid=0 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 client.15327 shutdown 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 msgr2=0x56461a1d8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feca0009b20 tx=0x7feca0005410 comp rx=0 tx=0).stop 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 shutdown_connections 2023-10-08T09:55:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x56461a238fe0 0x56461a235480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56461a1d21c0 0x56461a233850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fec8affd700 0 client.15327 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56461a1ca420 0x56461a1d86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 --2- 172.21.15.97:0/2901130103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461a1cbbc0 0x56461a1d8c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 >> 172.21.15.97:0/2901130103 conn(0x56461a13a210 msgr2=0x56461a1d3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 shutdown_connections 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.493+0000 7fecbe198b80 1 -- 172.21.15.97:0/2901130103 wait complete. 2023-10-08T09:55:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecbe198b80 20 client.15327 trim_cache size 0 max 16384 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24541, 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 55.312880006999997, 2023-10-08T09:55:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 98.520907445999995 2023-10-08T09:55:18.505 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 msgr2=0x7fec940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 0x7fec940a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fecac068910 tx=0x7feca0009260 comp rx=0 tx=0).stop 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 msgr2=0x7fecac068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00d980 tx=0x7fec9c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 shutdown_connections 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fec940a1500 0x7fec940a39c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecac101230 0x7fecac19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecac0ff6a0 0x7fecac068370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 --2- 172.21.15.97:0/3505898652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecac100890 0x7fecac19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 >> 172.21.15.97:0/3505898652 conn(0x7fecac0fac10 msgr2=0x7fecac103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 shutdown_connections 2023-10-08T09:55:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.494+0000 7fecb1faf700 1 -- 172.21.15.97:0/3505898652 wait complete. 2023-10-08T09:55:18.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T09:55:18.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1253331164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/909456740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1592924833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 -- 172.21.15.97:0/1484909159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 msgr2=0x7fad18101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1484909159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18101e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fad080099c0 tx=0x7fad08009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 -- 172.21.15.97:0/1484909159 shutdown_connections 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1484909159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18102350 0x7fad18108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1484909159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad181007c0 0x7fad18100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1484909159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18101e10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 -- 172.21.15.97:0/1484909159 >> 172.21.15.97:0/1484909159 conn(0x7fad180fbd30 msgr2=0x7fad180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 -- 172.21.15.97:0/1484909159 shutdown_connections 2023-10-08T09:55:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.630+0000 7fad1ce41700 1 -- 172.21.15.97:0/1484909159 wait complete. 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 Processor -- start 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 -- start start 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 0x7fad18196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad18102350 0x7fad1819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1819e230 con 0x7fad181019b0 2023-10-08T09:55:18.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1819e370 con 0x7fad181007c0 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1ce41700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1819e500 con 0x7fad18102350 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad15d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad15d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56328/0 (socket says 172.21.15.97:56328) 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad15d9b700 1 -- 172.21.15.97:0/1634118025 learned_addr learned my addr 172.21.15.97:0/1634118025 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad16d9d700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad18102350 0x7fad1819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.631+0000 7fad1659c700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 0x7fad18196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 -- 172.21.15.97:0/1634118025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad18102350 msgr2=0x7fad1819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad18102350 0x7fad1819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 -- 172.21.15.97:0/1634118025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 msgr2=0x7fad18196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 0x7fad18196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 -- 172.21.15.97:0/1634118025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080095f0 con 0x7fad181019b0 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad1659c700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 0x7fad18196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fad080036f0 tx=0x7fad080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad077fe700 1 -- 172.21.15.97:0/1634118025 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0801d070 con 0x7fad181019b0 2023-10-08T09:55:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1819e780 con 0x7fad181019b0 2023-10-08T09:55:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad077fe700 1 -- 172.21.15.97:0/1634118025 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad08004090 con 0x7fad181019b0 2023-10-08T09:55:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad077fe700 1 -- 172.21.15.97:0/1634118025 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08022d40 con 0x7fad181019b0 2023-10-08T09:55:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.632+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1819ec30 con 0x7fad181019b0 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.634+0000 7fad077fe700 1 -- 172.21.15.97:0/1634118025 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fad08022470 con 0x7fad181019b0 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.634+0000 7fad077fe700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 0x7fad000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.635+0000 7fad1659c700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 0x7fad000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.635+0000 7fad077fe700 1 -- 172.21.15.97:0/1634118025 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad080ce190 con 0x7fad181019b0 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.635+0000 7fad1659c700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 0x7fad000a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005950 tx=0x7fad0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.635+0000 7fad2902ab80 1 Processor -- start 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 -- start start 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 0x5568206dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556820645410 con 0x5568206de7b0 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556820636bd0 con 0x5568206dd010 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad2902ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568206344f0 con 0x5568206dd970 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56338/0 (socket says 172.21.15.97:56338) 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 -- 172.21.15.97:0/19837929 learned_addr learned my addr 172.21.15.97:0/19837929 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad1659c700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 0x5568206dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 -- 172.21.15.97:0/19837929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 msgr2=0x5568206de220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad15d9b700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206de220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 -- 172.21.15.97:0/19837929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 msgr2=0x5568206dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 0x5568206dd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 -- 172.21.15.97:0/19837929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55682045ad90 con 0x5568206de7b0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad1659c700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 0x5568206dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.636+0000 7fad16d9d700 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad100098f0 tx=0x7fad10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b163f9df719749 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad04ff9700 1 -- 172.21.15.97:0/19837929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10019070 con 0x5568206de7b0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad04ff9700 1 -- 172.21.15.97:0/19837929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad1000f040 con 0x5568206de7b0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad04ff9700 1 -- 172.21.15.97:0/19837929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10007bc0 con 0x5568206de7b0 2023-10-08T09:55:18.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 -- 172.21.15.97:0/19837929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 msgr2=0x5568206dec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad100098f0 tx=0x7fad10009c00 comp rx=0 tx=0).stop 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 -- 172.21.15.97:0/19837929 shutdown_connections 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206dec40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd010 0x5568206dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 --2- 172.21.15.97:0/19837929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206de220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 -- 172.21.15.97:0/19837929 >> 172.21.15.97:0/19837929 conn(0x556820651210 msgr2=0x5568206d7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 -- 172.21.15.97:0/19837929 shutdown_connections 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.637+0000 7fad2902ab80 1 -- 172.21.15.97:0/19837929 wait complete. 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 -- start start 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 Processor -- start 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 0x5568206ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682062fde0 con 0x5568206dd010 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568205cc6c0 con 0x5568206de7b0 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568205cc360 con 0x5568206dd970 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad2902ab80 10 client.0 _ll_register_callbacks cb 0x5568204297b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad1659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad1659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56340/0 (socket says 172.21.15.97:56340) 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad1659c700 1 -- 172.21.15.97:0/207321336 learned_addr learned my addr 172.21.15.97:0/207321336 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.638+0000 7fad16d9d700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 0x5568206ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad15d9b700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 -- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 msgr2=0x5568206e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206e73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 -- 172.21.15.97:0/207321336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 msgr2=0x5568206ee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 0x5568206ee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 -- 172.21.15.97:0/207321336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad10003c50 con 0x5568206dd010 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad16d9d700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 0x5568206ee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad15d9b700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad1659c700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00ff80 tx=0x7fad0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6314fe95625983f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c01e620 con 0x5568206dd010 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad0c01e780 con 0x5568206dd010 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568205f7e60 con 0x5568206dd010 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0c01e940 con 0x5568206dd010 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 10 client.15339 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556820443c10 con 0x5568206dd010 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568205f7e60 con 0x5568206dd010 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad0c024020 con 0x5568206dd010 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 10 client.15339 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 20 client.15339 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5568206f2990 0x5568206f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 4 client.15339 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.639+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556820628dc0 con 0x5568206f2990 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.640+0000 7fad15d9b700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5568206f2990 0x5568206f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.640+0000 7fad15d9b700 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5568206f2990 0x5568206f1450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75bc4ba7e3269793 server_cookie=9947e52f9911f700 in_seq=0 out_seq=0 2023-10-08T09:55:18.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.640+0000 7facfdffb700 10 client.15339 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:18.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.641+0000 7facfdffb700 1 -- 172.21.15.97:0/207321336 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556820628dc0 con 0x5568206f2990 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.641+0000 7facfdffb700 10 client.15339 handle_command_reply: tid=0 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.641+0000 7fad2902ab80 1 client.15339 shutdown 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 msgr2=0x5568206e6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00ff80 tx=0x7fad0c004750 comp rx=0 tx=0).stop 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 shutdown_connections 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5568206f2990 0x5568206f1450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206de7b0 0x5568206ee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7facfdffb700 0 client.15339 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206e6ea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 --2- 172.21.15.97:0/207321336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd970 0x5568206e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 >> 172.21.15.97:0/207321336 conn(0x556820651210 msgr2=0x5568206d7480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 shutdown_connections 2023-10-08T09:55:18.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 1 -- 172.21.15.97:0/207321336 wait complete. 2023-10-08T09:55:18.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.642+0000 7fad2902ab80 20 client.15339 trim_cache size 0 max 16384 2023-10-08T09:55:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 Processor -- start 2023-10-08T09:55:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 -- start start 2023-10-08T09:55:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd010 0x5568206f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568204b08d0 con 0x5568206de7b0 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568206d9920 con 0x5568206dd970 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad2902ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556820628dc0 con 0x5568206dd010 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.659+0000 7fad16d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56352/0 (socket says 172.21.15.97:56352) 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 -- 172.21.15.97:0/2952734391 learned_addr learned my addr 172.21.15.97:0/2952734391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad1659c700 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd010 0x5568206f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad15d9b700 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 -- 172.21.15.97:0/2952734391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd010 msgr2=0x5568206f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd010 0x5568206f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 -- 172.21.15.97:0/2952734391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 msgr2=0x5568206ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 -- 172.21.15.97:0/2952734391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556820481200 con 0x5568206de7b0 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad16d9d700 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad10004040 tx=0x7fad10004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21db590807f77a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad057fa700 1 -- 172.21.15.97:0/2952734391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10019030 con 0x5568206de7b0 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad057fa700 1 -- 172.21.15.97:0/2952734391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad10004db0 con 0x5568206de7b0 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.660+0000 7fad057fa700 1 -- 172.21.15.97:0/2952734391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10015480 con 0x5568206de7b0 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- 172.21.15.97:0/2952734391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 msgr2=0x5568206f1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad10004040 tx=0x7fad10004580 comp rx=0 tx=0).stop 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- 172.21.15.97:0/2952734391 shutdown_connections 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206de7b0 0x5568206f1990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206dd010 0x5568206f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- 172.21.15.97:0/2952734391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- 172.21.15.97:0/2952734391 >> 172.21.15.97:0/2952734391 conn(0x556820651210 msgr2=0x5568206e63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- 172.21.15.97:0/2952734391 shutdown_connections 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- 172.21.15.97:0/2952734391 wait complete. 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- start start 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 Processor -- start 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206de7b0 0x5568206ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568206ecee0 con 0x5568206dd010 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568206ed050 con 0x5568206dd970 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568206ed1c0 con 0x5568206de7b0 2023-10-08T09:55:18.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad2902ab80 10 client.0 _ll_register_callbacks cb 0x7fad0c00bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad1659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad1659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56354/0 (socket says 172.21.15.97:56354) 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad1659c700 1 -- 172.21.15.97:0/1517566977 learned_addr learned my addr 172.21.15.97:0/1517566977 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.661+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad16d9d700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206de7b0 0x5568206ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 -- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206de7b0 msgr2=0x5568206ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206de7b0 0x5568206ec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 -- 172.21.15.97:0/1517566977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 msgr2=0x5568206ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 -- 172.21.15.97:0/1517566977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55682045ad90 con 0x5568206dd010 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad1659c700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005a80 tx=0x7fad0c026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db594b9e9985aa73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00f9d0 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad0c039070 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad1000f040 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0c011b60 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 10 client.15345 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556820459260 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad1000f040 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad0c0114f0 con 0x5568206dd010 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 10 client.15345 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 10 client.15345 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 20 client.15345 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.662+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55682074c070 0x5568207484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad2902ab80 4 client.15345 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556820628ae0 con 0x55682074c070 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55682074c070 0x5568207484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7facfdffb700 10 client.15345 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad15d9b700 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55682074c070 0x5568207484b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=581168c35da5b3f2 server_cookie=b7c693ea3d1a8afc in_seq=0 out_seq=0 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7facfdffb700 1 -- 172.21.15.97:0/1517566977 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556820628ae0 con 0x55682074c070 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7facfdffb700 10 client.15345 handle_command_reply: tid=0 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad2902ab80 1 client.15345 shutdown 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 msgr2=0x5568206eb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.663+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005a80 tx=0x7fad0c026630 comp rx=0 tx=0).stop 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 shutdown_connections 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5568206de7b0 0x5568206ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568206dd010 0x5568206eb5e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55682074c070 0x5568207484b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 --2- 172.21.15.97:0/1517566977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5568206dd970 0x5568206ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7facfdffb700 0 client.15345 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 >> 172.21.15.97:0/1517566977 conn(0x556820651210 msgr2=0x5568206f1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 shutdown_connections 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 1 -- 172.21.15.97:0/1517566977 wait complete. 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad2902ab80 20 client.15345 trim_cache size 0 max 16384 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24562, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 56.492802230999999, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 94.490584489 2023-10-08T09:55:18.673 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 msgr2=0x7fad000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 0x7fad000a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005950 tx=0x7fad0c0058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 msgr2=0x7fad18196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fad080036f0 tx=0x7fad080047b0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 shutdown_connections 2023-10-08T09:55:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fad000a1480 0x7fad000a3940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad18102350 0x7fad1819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad181007c0 0x7fad18196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 --2- 172.21.15.97:0/1634118025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad181019b0 0x7fad18196c40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 >> 172.21.15.97:0/1634118025 conn(0x7fad180fbd30 msgr2=0x7fad18104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 shutdown_connections 2023-10-08T09:55:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.664+0000 7fad1ce41700 1 -- 172.21.15.97:0/1634118025 wait complete. 2023-10-08T09:55:18.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:55:18.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 -- 172.21.15.97:0/1546364137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a4102350 msgr2=0x7fd7a4108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/1546364137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a4102350 0x7fd7a4108c00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00b0d0 tx=0x7fd79c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 -- 172.21.15.97:0/1546364137 shutdown_connections 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/1546364137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a4102350 0x7fd7a4108c00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/1546364137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a41007c0 0x7fd7a4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/1546364137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 0x7fd7a4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 -- 172.21.15.97:0/1546364137 >> 172.21.15.97:0/1546364137 conn(0x7fd7a40fbd30 msgr2=0x7fd7a40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 -- 172.21.15.97:0/1546364137 shutdown_connections 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.797+0000 7fd7aa9da700 1 -- 172.21.15.97:0/1546364137 wait complete. 2023-10-08T09:55:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 Processor -- start 2023-10-08T09:55:18.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 -- start start 2023-10-08T09:55:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 0x7fd7a4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102350 0x7fd7a419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419e340 con 0x7fd7a41007c0 2023-10-08T09:55:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419e480 con 0x7fd7a4102350 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7aa9da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a419e610 con 0x7fd7a41019b0 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56368/0 (socket says 172.21.15.97:56368) 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7a3fff700 1 -- 172.21.15.97:0/2557014993 learned_addr learned my addr 172.21.15.97:0/2557014993 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.798+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 0x7fd7a4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102350 0x7fd7a419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 -- 172.21.15.97:0/2557014993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 msgr2=0x7fd7a4196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 0x7fd7a4196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 -- 172.21.15.97:0/2557014993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102350 msgr2=0x7fd7a419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102350 0x7fd7a419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 -- 172.21.15.97:0/2557014993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009d00 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd79400d980 tx=0x7fd79400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a17fa700 1 -- 172.21.15.97:0/2557014993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794004020 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a17fa700 1 -- 172.21.15.97:0/2557014993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd794009730 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a419e8f0 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7a17fa700 1 -- 172.21.15.97:0/2557014993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794003a80 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.799+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a419ee00 con 0x7fd7a41007c0 2023-10-08T09:55:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.801+0000 7fd7a17fa700 1 -- 172.21.15.97:0/2557014993 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd7940098a0 con 0x7fd7a41007c0 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.802+0000 7fd7a17fa700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 0x7fd78c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.802+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 0x7fd78c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.802+0000 7fd7a17fa700 1 -- 172.21.15.97:0/2557014993 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7940ccd20 con 0x7fd7a41007c0 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.802+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 0x7fd78c0a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd798005950 tx=0x7fd79800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 Processor -- start 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 -- start start 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 0x55bd2f0e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f04c410 con 0x55bd2f0e57b0 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f03dbd0 con 0x55bd2f0e4970 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f03b4f0 con 0x55bd2f0e4010 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56380/0 (socket says 172.21.15.97:56380) 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 -- 172.21.15.97:0/1814293627 learned_addr learned my addr 172.21.15.97:0/1814293627 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 -- 172.21.15.97:0/1814293627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 msgr2=0x55bd2f0e4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 0x55bd2f0e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 0x55bd2f0e4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 -- 172.21.15.97:0/1814293627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 msgr2=0x55bd2f0e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a8f77700 1 -- 172.21.15.97:0/1814293627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd2ee61d90 con 0x55bd2f0e57b0 2023-10-08T09:55:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.803+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 0x55bd2f0e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009c00 tx=0x7fd79c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff11f735eccfd5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd78a7fc700 1 -- 172.21.15.97:0/1814293627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00e070 con 0x55bd2f0e57b0 2023-10-08T09:55:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd78a7fc700 1 -- 172.21.15.97:0/1814293627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79c0090d0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd78a7fc700 1 -- 172.21.15.97:0/1814293627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005040 con 0x55bd2f0e57b0 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1814293627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 msgr2=0x55bd2f0e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009c00 tx=0x7fd79c009c30 comp rx=0 tx=0).stop 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1814293627 shutdown_connections 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0e5c40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4010 0x55bd2f0e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1814293627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1814293627 >> 172.21.15.97:0/1814293627 conn(0x55bd2f058210 msgr2=0x55bd2f0dedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1814293627 shutdown_connections 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.804+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1814293627 wait complete. 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 -- start start 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 Processor -- start 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f036de0 con 0x55bd2f0e4970 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2efd36c0 con 0x55bd2f0e4010 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2efd3360 con 0x55bd2f0e57b0 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7b6bc3b80 10 client.0 _ll_register_callbacks cb 0x55bd2ee307b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56386/0 (socket says 172.21.15.97:56386) 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7a37fe700 1 -- 172.21.15.97:0/2736048561 learned_addr learned my addr 172.21.15.97:0/2736048561 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.805+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 -- 172.21.15.97:0/2736048561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 msgr2=0x55bd2f0f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 -- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 msgr2=0x55bd2f0edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0edcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 -- 172.21.15.97:0/2736048561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a4109550 con 0x55bd2f0e4970 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd798011770 tx=0x7fd7980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9fbd7d2227d71f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798010ee0 con 0x55bd2f0e4970 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd798017070 con 0x55bd2f0e4970 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd2effee60 con 0x55bd2f0e4970 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd79801f840 con 0x55bd2f0e4970 2023-10-08T09:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7b6bc3b80 10 client.15357 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd2ee4ac10 con 0x55bd2f0e4970 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd2effee60 con 0x55bd2f0e4970 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd798028b70 con 0x55bd2f0e4970 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.806+0000 7fd7b6bc3b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7b6bc3b80 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7b6bc3b80 20 client.15357 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f0f99a0 0x55bd2f0f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7b6bc3b80 4 client.15357 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd2f02fdc0 con 0x55bd2f0f99a0 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f0f99a0 0x55bd2f0f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7837fe700 10 client.15357 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:18.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.807+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f0f99a0 0x55bd2f0f8430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd871f978e4f40b server_cookie=164eef4e5746b0ba in_seq=0 out_seq=0 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7837fe700 1 -- 172.21.15.97:0/2736048561 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bd2f02fdc0 con 0x55bd2f0f99a0 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7837fe700 10 client.15357 handle_command_reply: tid=0 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 client.15357 shutdown 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 msgr2=0x55bd2f0ee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd798011770 tx=0x7fd7980047c0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 shutdown_connections 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f0f99a0 0x55bd2f0f8430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7837fe700 0 client.15357 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/2736048561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4970 0x55bd2f0ee230 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 >> 172.21.15.97:0/2736048561 conn(0x55bd2f058210 msgr2=0x55bd2f0de480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 shutdown_connections 2023-10-08T09:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/2736048561 wait complete. 2023-10-08T09:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.808+0000 7fd7b6bc3b80 20 client.15357 trim_cache size 0 max 16384 2023-10-08T09:55:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 Processor -- start 2023-10-08T09:55:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 -- start start 2023-10-08T09:55:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2eeb78d0 con 0x55bd2f0e4010 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f0ec880 con 0x55bd2f0e4970 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f02fdf0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56390/0 (socket says 172.21.15.97:56390) 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 -- 172.21.15.97:0/1974844867 learned_addr learned my addr 172.21.15.97:0/1974844867 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 -- 172.21.15.97:0/1974844867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 msgr2=0x55bd2f0ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 -- 172.21.15.97:0/1974844867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 msgr2=0x55bd2f0f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 -- 172.21.15.97:0/1974844867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd2ee88200 con 0x55bd2f0e4010 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.826+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd794096230 tx=0x7fd794099b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e1ade82d3a99e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd78affd700 1 -- 172.21.15.97:0/1974844867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940a20c0 con 0x55bd2f0e4010 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd78affd700 1 -- 172.21.15.97:0/1974844867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7940ca620 con 0x55bd2f0e4010 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd78affd700 1 -- 172.21.15.97:0/1974844867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940a1c30 con 0x55bd2f0e4010 2023-10-08T09:55:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1974844867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 msgr2=0x55bd2f0ed200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd794096230 tx=0x7fd794099b80 comp rx=0 tx=0).stop 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1974844867 shutdown_connections 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e4010 0x55bd2f0ed200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/1974844867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1974844867 >> 172.21.15.97:0/1974844867 conn(0x55bd2f058210 msgr2=0x55bd2f0daee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1974844867 shutdown_connections 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/1974844867 wait complete. 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.827+0000 7fd7b6bc3b80 1 -- start start 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 Processor -- start 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f0f52f0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f0f5460 con 0x55bd2f0e4010 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2f0f5620 con 0x55bd2f0e4970 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7b6bc3b80 10 client.0 _ll_register_callbacks cb 0x7fd798010d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a8f77700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a8f77700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56400/0 (socket says 172.21.15.97:56400) 2023-10-08T09:55:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a8f77700 1 -- 172.21.15.97:0/3180660826 learned_addr learned my addr 172.21.15.97:0/3180660826 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a37fe700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a8f77700 1 -- 172.21.15.97:0/3180660826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4970 msgr2=0x55bd2f0f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.828+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7a8f77700 1 -- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 msgr2=0x55bd2f0f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7a8f77700 1 -- 172.21.15.97:0/3180660826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a419ee00 con 0x55bd2f0e57b0 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7a8f77700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006130 tx=0x7fd79c01b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e042364a34ef478f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00e040 con 0x55bd2f0e57b0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd79c01b9e0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd79c0090d0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd79c00b850 con 0x55bd2f0e57b0 2023-10-08T09:55:18.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 10 client.15363 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd2ee60260 con 0x55bd2f0e57b0 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79c0090d0 con 0x55bd2f0e57b0 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd79c035070 con 0x55bd2f0e57b0 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:55:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 20 client.15363 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f1521e0 0x55bd2f14e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 4 client.15363 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.829+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bd2f02fae0 con 0x55bd2f1521e0 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f1521e0 0x55bd2f14e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7a3fff700 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f1521e0 0x55bd2f14e720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f350fb26c5f159fe server_cookie=b40624672e6a009f in_seq=0 out_seq=0 2023-10-08T09:55:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7837fe700 10 client.15363 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7837fe700 1 -- 172.21.15.97:0/3180660826 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55bd2f02fae0 con 0x55bd2f1521e0 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7837fe700 10 client.15363 handle_command_reply: tid=0 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7b6bc3b80 1 client.15363 shutdown 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 msgr2=0x55bd2f0f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.830+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006130 tx=0x7fd79c01b660 comp rx=0 tx=0).stop 2023-10-08T09:55:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 shutdown_connections 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bd2f0e57b0 0x55bd2f0f4c50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd2f0e4010 0x55bd2f0f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bd2f1521e0 0x55bd2f14e720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 --2- 172.21.15.97:0/3180660826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd2f0e4970 0x55bd2f0f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 >> 172.21.15.97:0/3180660826 conn(0x55bd2f058210 msgr2=0x55bd2f0ed200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7837fe700 0 client.15363 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 shutdown_connections 2023-10-08T09:55:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 1 -- 172.21.15.97:0/3180660826 wait complete. 2023-10-08T09:55:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7b6bc3b80 20 client.15363 trim_cache size 0 max 16384 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24566, 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 81.631059238999995, 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 87.777100587999996 2023-10-08T09:55:18.853 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 msgr2=0x7fd78c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 0x7fd78c0a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd798005950 tx=0x7fd79800b410 comp rx=0 tx=0).stop 2023-10-08T09:55:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 msgr2=0x7fd7a4196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd79400d980 tx=0x7fd79400dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 shutdown_connections 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd78c0a1480 0x7fd78c0a3940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7a4102350 0x7fd7a419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7a41007c0 0x7fd7a4196810 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 --2- 172.21.15.97:0/2557014993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7a41019b0 0x7fd7a4196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:18.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 >> 172.21.15.97:0/2557014993 conn(0x7fd7a40fbd30 msgr2=0x7fd7a4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:18.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 shutdown_connections 2023-10-08T09:55:18.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:18.831+0000 7fd7aa9da700 1 -- 172.21.15.97:0/2557014993 wait complete. 2023-10-08T09:55:18.861 INFO:tasks.cephadm:Setup complete, yielding 2023-10-08T09:55:18.862 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T09:55:18.874 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:55:18.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-08T09:55:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:19 smithi097 ceph-mon[99491]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:19 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/154792861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:19 smithi178 ceph-mon[103156]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:19 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/154792861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:19 smithi186 ceph-mon[104393]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:19 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/154792861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 -- 172.21.15.97:0/3781935706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 msgr2=0x7f9b80100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 --2- 172.21.15.97:0/3781935706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80100da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b700099c0 tx=0x7f9b70009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 -- 172.21.15.97:0/3781935706 shutdown_connections 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 --2- 172.21.15.97:0/3781935706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 --2- 172.21.15.97:0/3781935706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b800ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 --2- 172.21.15.97:0/3781935706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80100da0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 -- 172.21.15.97:0/3781935706 >> 172.21.15.97:0/3781935706 conn(0x7f9b800facc0 msgr2=0x7f9b800fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 -- 172.21.15.97:0/3781935706 shutdown_connections 2023-10-08T09:55:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.659+0000 7f9b867f5700 1 -- 172.21.15.97:0/3781935706 wait complete. 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 Processor -- start 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 -- start start 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b80103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80102a00 con 0x7f9b80100940 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80102b70 con 0x7f9b800ff740 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b867f5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801a07a0 con 0x7f9b801012e0 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56412/0 (socket says 172.21.15.97:56412) 2023-10-08T09:55:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 -- 172.21.15.97:0/2201155426 learned_addr learned my addr 172.21.15.97:0/2201155426 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b84d92700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7ffff700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b80103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 -- 172.21.15.97:0/2201155426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 msgr2=0x7f9b80102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 -- 172.21.15.97:0/2201155426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 msgr2=0x7f9b80103680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b80103680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7f7fe700 1 -- 172.21.15.97:0/2201155426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700095f0 con 0x7f9b80100940 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b84d92700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.660+0000 7f9b7ffff700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b80103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b7f7fe700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9b700036f0 tx=0x7f9b70003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7001d070 con 0x7f9b80100940 2023-10-08T09:55:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b70022470 con 0x7f9b80100940 2023-10-08T09:55:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b801a09d0 con 0x7f9b80100940 2023-10-08T09:55:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000f6e0 con 0x7f9b80100940 2023-10-08T09:55:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.661+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b801a0e70 con 0x7f9b80100940 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.662+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8004e4c0 con 0x7f9b80100940 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.663+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9b70022890 con 0x7f9b80100940 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.663+0000 7f9b7d7fa700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 0x7f9b6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.663+0000 7f9b7ffff700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 0x7f9b6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.663+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b700ce6a0 con 0x7f9b80100940 2023-10-08T09:55:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.664+0000 7f9b7ffff700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 0x7f9b6c0a39c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b801007a0 tx=0x7f9b68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:20.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.667+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9b70097a20 con 0x7f9b80100940 2023-10-08T09:55:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.876+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b800fcf10 con 0x7f9b6c0a1500 2023-10-08T09:55:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.877+0000 7f9b7d7fa700 1 -- 172.21.15.97:0/2201155426 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9b800fcf10 con 0x7f9b6c0a1500 2023-10-08T09:55:20.878 INFO:teuthology.orchestra.run.smithi097.stdout:Backend: cephadm 2023-10-08T09:55:20.878 INFO:teuthology.orchestra.run.smithi097.stdout:Available: Yes 2023-10-08T09:55:20.878 INFO:teuthology.orchestra.run.smithi097.stdout:Paused: No 2023-10-08T09:55:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.879+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 msgr2=0x7f9b6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.879+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 0x7f9b6c0a39c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b801007a0 tx=0x7f9b68005f90 comp rx=0 tx=0).stop 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 msgr2=0x7f9b80101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9b700036f0 tx=0x7f9b70003970 comp rx=0 tx=0).stop 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 shutdown_connections 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b6c0a1500 0x7f9b6c0a39c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b801012e0 0x7f9b80102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b800ff740 0x7f9b80103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 --2- 172.21.15.97:0/2201155426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b80100940 0x7f9b80101d20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 >> 172.21.15.97:0/2201155426 conn(0x7f9b800facc0 msgr2=0x7f9b800fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 shutdown_connections 2023-10-08T09:55:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:20.880+0000 7f9b867f5700 1 -- 172.21.15.97:0/2201155426 wait complete. 2023-10-08T09:55:21.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-08T09:55:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:21 smithi097 ceph-mon[99491]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:21 smithi178 ceph-mon[103156]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:21.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:21 smithi186 ceph-mon[104393]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:21.849 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:22.378 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:22 smithi097 ceph-mon[99491]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:22.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:22 smithi178 ceph-mon[103156]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:22.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:22 smithi186 ceph-mon[104393]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.046+0000 7fe62243c700 1 -- 172.21.15.97:0/2345786198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c0ff740 msgr2=0x7fe61c0ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.046+0000 7fe62243c700 1 --2- 172.21.15.97:0/2345786198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c0ff740 0x7fe61c0ffb60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0099c0 tx=0x7fe60c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 -- 172.21.15.97:0/2345786198 shutdown_connections 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 --2- 172.21.15.97:0/2345786198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c1012e0 0x7fe61c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 --2- 172.21.15.97:0/2345786198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c0ff740 0x7fe61c0ffb60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 --2- 172.21.15.97:0/2345786198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c100940 0x7fe61c100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 -- 172.21.15.97:0/2345786198 >> 172.21.15.97:0/2345786198 conn(0x7fe61c0fad00 msgr2=0x7fe61c0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 -- 172.21.15.97:0/2345786198 shutdown_connections 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.047+0000 7fe62243c700 1 -- 172.21.15.97:0/2345786198 wait complete. 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 Processor -- start 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 -- start start 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c0ff740 0x7fe61c1956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 0x7fe61c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c19d220 con 0x7fe61c100940 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c19d360 con 0x7fe61c1012e0 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe62243c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c19d4f0 con 0x7fe61c0ff740 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56424/0 (socket says 172.21.15.97:56424) 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 -- 172.21.15.97:0/825490123 learned_addr learned my addr 172.21.15.97:0/825490123 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61bfff700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c0ff740 0x7fe61c1956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe6209d9700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 0x7fe61c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 -- 172.21.15.97:0/825490123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c0ff740 msgr2=0x7fe61c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c0ff740 0x7fe61c1956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 -- 172.21.15.97:0/825490123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 msgr2=0x7fe61c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 0x7fe61c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe61b7fe700 1 -- 172.21.15.97:0/825490123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0095f0 con 0x7fe61c100940 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.048+0000 7fe6209d9700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 0x7fe61c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe61b7fe700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c950 tx=0x7fe61000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61000cea0 con 0x7fe61c100940 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe610004830 con 0x7fe61c100940 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c19d7d0 con 0x7fe61c100940 2023-10-08T09:55:23.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100054e0 con 0x7fe61c100940 2023-10-08T09:55:23.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.049+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c19dcd0 con 0x7fe61c100940 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.050+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c106bc0 con 0x7fe61c100940 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.051+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe6100049a0 con 0x7fe61c100940 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.051+0000 7fe6197fa700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 0x7fe6040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.052+0000 7fe61bfff700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 0x7fe6040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.052+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6100cd070 con 0x7fe61c100940 2023-10-08T09:55:23.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.052+0000 7fe61bfff700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 0x7fe6040a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe60c003820 tx=0x7fe60c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:23.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.055+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe6100963c0 con 0x7fe61c100940 2023-10-08T09:55:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.255+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe61c060c30 con 0x7fe6040a1480 2023-10-08T09:55:23.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.268+0000 7fe6197fa700 1 -- 172.21.15.97:0/825490123 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe61c060c30 con 0x7fe6040a1480 2023-10-08T09:55:23.269 INFO:teuthology.orchestra.run.smithi097.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-08T09:55:23.269 INFO:teuthology.orchestra.run.smithi097.stdout:mds.a smithi097 running (94s) 15s ago 94s 14.7M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 46fea3fdc95e 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.b smithi178 running (92s) 22s ago 92s 24.7M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 9c00c6a5696f 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.c smithi186 running (103s) 23s ago 103s 20.6M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 8287837bdc2c 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.d smithi097 running (110s) 15s ago 109s 17.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 5d6d68cbb500 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.e smithi178 running (108s) 22s ago 107s 19.4M - 18.0.0-6639-g7b3143ba 09a1f3752b5d dd8884e0d666 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.f smithi186 running (105s) 23s ago 105s 19.5M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 36708fdd089d 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.g smithi097 running (97s) 15s ago 96s 13.7M - 18.0.0-6639-g7b3143ba 09a1f3752b5d d49f20626efd 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.h smithi178 running (101s) 22s ago 101s 17.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d e6ebdff16e14 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.i smithi186 running (99s) 23s ago 99s 17.9M - 18.0.0-6639-g7b3143ba 09a1f3752b5d bb9adb0f4832 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.j smithi097 running (112s) 15s ago 112s 16.1M - 18.0.0-6639-g7b3143ba 09a1f3752b5d cfa9ad3e6843 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.k smithi178 running (114s) 22s ago 114s 14.8M - 18.0.0-6639-g7b3143ba 09a1f3752b5d fbdf38855a4f 2023-10-08T09:55:23.270 INFO:teuthology.orchestra.run.smithi097.stdout:mds.l smithi186 running (116s) 23s ago 116s 19.2M - 18.0.0-6639-g7b3143ba 09a1f3752b5d d563dcb7c40a 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.x smithi097 *:9283,8765 running (8m) 15s ago 8m 521M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 931644c8953d 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.y smithi178 *:8443,8765 running (6m) 22s ago 6m 450M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 56e509f49bff 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.z smithi186 *:8443,8765 running (6m) 23s ago 6m 452M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 5a9c483f2de4 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a smithi097 running (8m) 15s ago 8m 95.0M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d b752fcf3b8f7 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b smithi178 running (6m) 22s ago 6m 59.9M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d 38e4b0bdddca 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c smithi186 running (7m) 23s ago 7m 59.2M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d 4751f01d21e5 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 smithi097 running (5m) 15s ago 5m 71.6M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d f172d76beca6 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 smithi178 running (5m) 22s ago 5m 74.2M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 81023a7ffd2e 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 smithi186 running (5m) 23s ago 5m 72.3M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d f269a6644e68 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 smithi097 running (4m) 15s ago 4m 63.1M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 65a71e678b5e 2023-10-08T09:55:23.271 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 smithi178 running (4m) 22s ago 4m 61.5M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 6fe5bdfdc5c2 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 smithi186 running (4m) 23s ago 4m 68.1M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 274f73ff9487 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 smithi097 running (3m) 15s ago 3m 62.7M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d fa77fcaf3651 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 smithi178 running (3m) 22s ago 3m 64.6M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d afc542a9fe08 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 smithi186 running (3m) 23s ago 3m 61.1M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d eba5cec714c2 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 smithi097 running (2m) 15s ago 2m 60.1M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d bfc86ad0e099 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 smithi178 running (2m) 22s ago 2m 61.6M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d acf9ef4aa886 2023-10-08T09:55:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 smithi186 running (2m) 23s ago 2m 57.4M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d ff7a8ecd6f5c 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 msgr2=0x7fe6040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 0x7fe6040a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe60c003820 tx=0x7fe60c005ec0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 msgr2=0x7fe61c195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c950 tx=0x7fe61000cc60 comp rx=0 tx=0).stop 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 shutdown_connections 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6040a1480 0x7fe6040a3940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe61c1012e0 0x7fe61c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe61c0ff740 0x7fe61c1956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 --2- 172.21.15.97:0/825490123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe61c100940 0x7fe61c195c20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.270+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 >> 172.21.15.97:0/825490123 conn(0x7fe61c0fad00 msgr2=0x7fe61c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.271+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 shutdown_connections 2023-10-08T09:55:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:23.271+0000 7fe62243c700 1 -- 172.21.15.97:0/825490123 wait complete. 2023-10-08T09:55:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:23 smithi097 ceph-mon[99491]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:23 smithi178 ceph-mon[103156]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:23 smithi186 ceph-mon[104393]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:23.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-08T09:55:24.268 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:25 smithi097 ceph-mon[99491]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:25.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:25 smithi097 ceph-mon[99491]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:25 smithi178 ceph-mon[103156]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:25 smithi178 ceph-mon[103156]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 -- 172.21.15.97:0/3839824491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 msgr2=0x7f5b48100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 --2- 172.21.15.97:0/3839824491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48100d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b380099c0 tx=0x7f5b38009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 -- 172.21.15.97:0/3839824491 shutdown_connections 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 --2- 172.21.15.97:0/3839824491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b481012d0 0x7f5b48107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 --2- 172.21.15.97:0/3839824491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b480ff730 0x7f5b480ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 --2- 172.21.15.97:0/3839824491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48100d90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 -- 172.21.15.97:0/3839824491 >> 172.21.15.97:0/3839824491 conn(0x7f5b480facf0 msgr2=0x7f5b480fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 -- 172.21.15.97:0/3839824491 shutdown_connections 2023-10-08T09:55:25.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 -- 172.21.15.97:0/3839824491 wait complete. 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.472+0000 7f5b502aa700 1 Processor -- start 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 -- start start 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b480ff730 0x7f5b48103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b481012d0 0x7f5b48102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48102be0 con 0x7f5b48100930 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48102d50 con 0x7f5b480ff730 2023-10-08T09:55:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b481a07b0 con 0x7f5b481012d0 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56438/0 (socket says 172.21.15.97:56438) 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 -- 172.21.15.97:0/393470171 learned_addr learned my addr 172.21.15.97:0/393470171 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4e847700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b481012d0 0x7f5b48102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4e046700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b480ff730 0x7f5b48103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 -- 172.21.15.97:0/393470171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b481012d0 msgr2=0x7f5b48102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b481012d0 0x7f5b48102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 -- 172.21.15.97:0/393470171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b480ff730 msgr2=0x7f5b48103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b480ff730 0x7f5b48103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 -- 172.21.15.97:0/393470171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380095f0 con 0x7f5b48100930 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b4d845700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b380036f0 tx=0x7f5b380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:25.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3801d070 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b38004090 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b481a09e0 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.473+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38022d40 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.474+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b481a0ed0 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.474+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4804e4c0 con 0x7f5b48100930 2023-10-08T09:55:25.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.476+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5b38022470 con 0x7f5b48100930 2023-10-08T09:55:25.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.476+0000 7f5b3f7fe700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 0x7f5b340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:25.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.476+0000 7f5b4e046700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 0x7f5b340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:25.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.476+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b380ce490 con 0x7f5b48100930 2023-10-08T09:55:25.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.476+0000 7f5b4e046700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 0x7f5b340a39c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005950 tx=0x7f5b440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:25.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.480+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5b38097870 con 0x7f5b48100930 2023-10-08T09:55:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:25 smithi186 ceph-mon[104393]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:25 smithi186 ceph-mon[104393]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.677+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b480688a0 con 0x7f5b340a1500 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.683+0000 7f5b3f7fe700 1 -- 172.21.15.97:0/393470171 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f5b480688a0 con 0x7f5b340a1500 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stdout:mds.all 12/12 25s ago 2m smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi178=b;smithi178=e;smithi178=h;smithi178=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stdout:mgr 3/3 25s ago 6m smithi097=x;smithi178=y;smithi186=z;count:3 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stdout:mon 3/3 25s ago 7m smithi097:172.21.15.97=a;smithi178:172.21.15.178=b;smithi186:172.21.15.186=c;count:3 2023-10-08T09:55:25.684 INFO:teuthology.orchestra.run.smithi097.stdout:osd 12 25s ago - 2023-10-08T09:55:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.685+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 msgr2=0x7f5b340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.685+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 0x7f5b340a39c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005950 tx=0x7f5b440058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.685+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 msgr2=0x7f5b48101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.685+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b380036f0 tx=0x7f5b380047b0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 shutdown_connections 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5b340a1500 0x7f5b340a39c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b481012d0 0x7f5b48102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b480ff730 0x7f5b48103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 --2- 172.21.15.97:0/393470171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b48100930 0x7f5b48101f00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 >> 172.21.15.97:0/393470171 conn(0x7f5b480facf0 msgr2=0x7f5b48068190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 shutdown_connections 2023-10-08T09:55:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:25.686+0000 7f5b502aa700 1 -- 172.21.15.97:0/393470171 wait complete. 2023-10-08T09:55:26.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-08T09:55:26.676 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:27.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:27 smithi097 ceph-mon[99491]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:27.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:27 smithi097 ceph-mon[99491]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:27 smithi178 ceph-mon[103156]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:27 smithi178 ceph-mon[103156]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:27 smithi186 ceph-mon[104393]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:27 smithi186 ceph-mon[104393]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:27.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 -- 172.21.15.97:0/3835672325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c180ff730 msgr2=0x7f2c180ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:27.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 --2- 172.21.15.97:0/3835672325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c180ff730 0x7f2c180ffb50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c140099c0 tx=0x7f2c14009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 -- 172.21.15.97:0/3835672325 shutdown_connections 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 --2- 172.21.15.97:0/3835672325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 0x7f2c18107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 --2- 172.21.15.97:0/3835672325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c180ff730 0x7f2c180ffb50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 --2- 172.21.15.97:0/3835672325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c18100930 0x7f2c18100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 -- 172.21.15.97:0/3835672325 >> 172.21.15.97:0/3835672325 conn(0x7f2c180facf0 msgr2=0x7f2c180fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 -- 172.21.15.97:0/3835672325 shutdown_connections 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.968+0000 7f2c20464700 1 -- 172.21.15.97:0/3835672325 wait complete. 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 Processor -- start 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 -- start start 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c180ff730 0x7f2c18068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 0x7f2c1819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1819e210 con 0x7f2c18100930 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1819e380 con 0x7f2c180ff730 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c20464700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1819e540 con 0x7f2c181012d0 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56452/0 (socket says 172.21.15.97:56452) 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 -- 172.21.15.97:0/3504945372 learned_addr learned my addr 172.21.15.97:0/3504945372 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1ea01700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 0x7f2c1819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1e200700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c180ff730 0x7f2c18068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 -- 172.21.15.97:0/3504945372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 msgr2=0x7f2c1819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 0x7f2c1819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 -- 172.21.15.97:0/3504945372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c180ff730 msgr2=0x7f2c18068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c180ff730 0x7f2c18068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 -- 172.21.15.97:0/3504945372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c140095f0 con 0x7f2c18100930 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.969+0000 7f2c1d9ff700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800d950 tx=0x7f2c0800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.970+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08009b40 con 0x7f2c18100930 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.970+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c08009ca0 con 0x7f2c18100930 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.970+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800f620 con 0x7f2c18100930 2023-10-08T09:55:27.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.970+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1819e820 con 0x7f2c18100930 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.970+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1819ecd0 con 0x7f2c18100930 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.971+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c18106c10 con 0x7f2c18100930 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.972+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2c08010460 con 0x7f2c18100930 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.972+0000 7f2c0f7fe700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 0x7f2c040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.972+0000 7f2c1e200700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 0x7f2c040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.972+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2c080cd300 con 0x7f2c18100930 2023-10-08T09:55:27.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.973+0000 7f2c1e200700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 0x7f2c040a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400b580 tx=0x7f2c14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:27.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:27.976+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2c080966f0 con 0x7f2c18100930 2023-10-08T09:55:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.170+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c1802cf80 con 0x7f2c040a1480 2023-10-08T09:55:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.172+0000 7f2c0f7fe700 1 -- 172.21.15.97:0/3504945372 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f2c1802cf80 con 0x7f2c040a1480 2023-10-08T09:55:28.172 INFO:teuthology.orchestra.run.smithi097.stdout:HOST ADDR LABELS STATUS 2023-10-08T09:55:28.173 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 172.21.15.97 2023-10-08T09:55:28.173 INFO:teuthology.orchestra.run.smithi097.stdout:smithi178 172.21.15.178 2023-10-08T09:55:28.173 INFO:teuthology.orchestra.run.smithi097.stdout:smithi186 172.21.15.186 2023-10-08T09:55:28.173 INFO:teuthology.orchestra.run.smithi097.stdout:3 hosts in cluster 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 msgr2=0x7f2c040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 0x7f2c040a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400b580 tx=0x7f2c14005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 msgr2=0x7f2c1819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800d950 tx=0x7f2c0800dd10 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 shutdown_connections 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c040a1480 0x7f2c040a3940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c181012d0 0x7f2c1819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c180ff730 0x7f2c18068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 --2- 172.21.15.97:0/3504945372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c18100930 0x7f2c1819b4a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:28.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 >> 172.21.15.97:0/3504945372 conn(0x7f2c180facf0 msgr2=0x7f2c18103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:28.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 shutdown_connections 2023-10-08T09:55:28.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:28.174+0000 7f2c20464700 1 -- 172.21.15.97:0/3504945372 wait complete. 2023-10-08T09:55:28.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-08T09:55:29.184 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:29.462 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:29 smithi097 ceph-mon[99491]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:29.463 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:29 smithi097 ceph-mon[99491]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:29 smithi186 ceph-mon[104393]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:29 smithi186 ceph-mon[104393]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:29.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:29 smithi178 ceph-mon[103156]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:29.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:29 smithi178 ceph-mon[103156]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:30.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 -- 172.21.15.97:0/3913852665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c1012d0 msgr2=0x7fef7c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:30.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 --2- 172.21.15.97:0/3913852665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c1012d0 0x7fef7c107b80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fef7400b0d0 tx=0x7fef7400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 -- 172.21.15.97:0/3913852665 shutdown_connections 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 --2- 172.21.15.97:0/3913852665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c1012d0 0x7fef7c107b80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 --2- 172.21.15.97:0/3913852665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c0ff730 0x7fef7c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 --2- 172.21.15.97:0/3913852665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c100930 0x7fef7c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 -- 172.21.15.97:0/3913852665 >> 172.21.15.97:0/3913852665 conn(0x7fef7c0facf0 msgr2=0x7fef7c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 -- 172.21.15.97:0/3913852665 shutdown_connections 2023-10-08T09:55:30.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.388+0000 7fef80a40700 1 -- 172.21.15.97:0/3913852665 wait complete. 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 Processor -- start 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 -- start start 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c100930 0x7fef7c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 0x7fef7c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c19d2c0 con 0x7fef7c0ff730 2023-10-08T09:55:30.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c19d400 con 0x7fef7c1012d0 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef80a40700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c19d590 con 0x7fef7c100930 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.389+0000 7fef7a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56462/0 (socket says 172.21.15.97:56462) 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 -- 172.21.15.97:0/3494244977 learned_addr learned my addr 172.21.15.97:0/3494244977 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef79d9b700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c100930 0x7fef7c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7ad9d700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 0x7fef7c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 -- 172.21.15.97:0/3494244977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c100930 msgr2=0x7fef7c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c100930 0x7fef7c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 -- 172.21.15.97:0/3494244977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 msgr2=0x7fef7c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 0x7fef7c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 -- 172.21.15.97:0/3494244977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009d00 con 0x7fef7c0ff730 2023-10-08T09:55:30.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7ad9d700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 0x7fef7c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef7a59c700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef70009920 tx=0x7fef70009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70012070 con 0x7fef7c0ff730 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef700044e0 con 0x7fef7c0ff730 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c0681f0 con 0x7fef7c0ff730 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70007ce0 con 0x7fef7c0ff730 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.390+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c068740 con 0x7fef7c0ff730 2023-10-08T09:55:30.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.391+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c04e4c0 con 0x7fef7c0ff730 2023-10-08T09:55:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.392+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fef70016030 con 0x7fef7c0ff730 2023-10-08T09:55:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.393+0000 7fef6b7fe700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 0x7fef640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.393+0000 7fef79d9b700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 0x7fef640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.393+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fef700ce1c0 con 0x7fef7c0ff730 2023-10-08T09:55:30.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.394+0000 7fef79d9b700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 0x7fef640a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:30.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.398+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef70097510 con 0x7fef7c0ff730 2023-10-08T09:55:30.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.591+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fef7c106660 con 0x7fef640a1500 2023-10-08T09:55:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.593+0000 7fef6b7fe700 1 -- 172.21.15.97:0/3494244977 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fef7c106660 con 0x7fef640a1500 2023-10-08T09:55:30.593 INFO:teuthology.orchestra.run.smithi097.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-08T09:55:30.594 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T09:55:30.594 INFO:teuthology.orchestra.run.smithi097.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-08T09:55:30.594 INFO:teuthology.orchestra.run.smithi097.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 msgr2=0x7fef640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 0x7fef640a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 msgr2=0x7fef7c195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef70009920 tx=0x7fef70009ce0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 shutdown_connections 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fef640a1500 0x7fef640a39c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c1012d0 0x7fef7c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c0ff730 0x7fef7c195780 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 --2- 172.21.15.97:0/3494244977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef7c100930 0x7fef7c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 >> 172.21.15.97:0/3494244977 conn(0x7fef7c0facf0 msgr2=0x7fef7c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:30.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 shutdown_connections 2023-10-08T09:55:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:30.595+0000 7fef80a40700 1 -- 172.21.15.97:0/3494244977 wait complete. 2023-10-08T09:55:31.039 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T09:55:31.049 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:55:31.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-08T09:55:31.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:31 smithi097 ceph-mon[99491]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:31 smithi186 ceph-mon[104393]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:31.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:31 smithi178 ceph-mon[103156]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:32.389 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:32 smithi097 ceph-mon[99491]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:32.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:32 smithi186 ceph-mon[104393]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:32.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:32 smithi178 ceph-mon[103156]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 -- 172.21.15.97:0/4197742012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c100840 msgr2=0x7f007c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 --2- 172.21.15.97:0/4197742012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c100840 0x7f007c100c60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f006c0099c0 tx=0x7f006c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 -- 172.21.15.97:0/4197742012 shutdown_connections 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 --2- 172.21.15.97:0/4197742012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c1023e0 0x7f007c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 --2- 172.21.15.97:0/4197742012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c100840 0x7f007c100c60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 --2- 172.21.15.97:0/4197742012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 0x7f007c101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 -- 172.21.15.97:0/4197742012 >> 172.21.15.97:0/4197742012 conn(0x7f007c0fbe00 msgr2=0x7f007c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 -- 172.21.15.97:0/4197742012 shutdown_connections 2023-10-08T09:55:32.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.849+0000 7f00825ea700 1 -- 172.21.15.97:0/4197742012 wait complete. 2023-10-08T09:55:32.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.850+0000 7f00825ea700 1 Processor -- start 2023-10-08T09:55:32.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.850+0000 7f00825ea700 1 -- start start 2023-10-08T09:55:32.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.850+0000 7f00825ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 0x7f007c1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.850+0000 7f00825ea700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 0x7f007c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.850+0000 7f00825ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00825ea700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19e2f0 con 0x7f007c1023e0 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00825ea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19e430 con 0x7f007c100840 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00825ea700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19e5c0 con 0x7f007c101a40 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56478/0 (socket says 172.21.15.97:56478) 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 -- 172.21.15.97:0/2533278618 learned_addr learned my addr 172.21.15.97:0/2533278618 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f007b7fe700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 0x7f007c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f007bfff700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 0x7f007c1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:32.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 -- 172.21.15.97:0/2533278618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 msgr2=0x7f007c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 0x7f007c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 -- 172.21.15.97:0/2533278618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 msgr2=0x7f007c1967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 0x7f007c1967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 -- 172.21.15.97:0/2533278618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c0095f0 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f007bfff700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 0x7f007c1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f0080b87700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f007400b9a0 tx=0x7f007400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400c930 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0074014440 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c19e8a0 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740075d0 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.851+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c19ed50 con 0x7f007c1023e0 2023-10-08T09:55:32.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.852+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c107d20 con 0x7f007c1023e0 2023-10-08T09:55:32.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.853+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f00740145b0 con 0x7f007c1023e0 2023-10-08T09:55:32.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.854+0000 7f00797fa700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 0x7f00640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:32.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.854+0000 7f007bfff700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 0x7f00640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.854+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0074006e40 con 0x7f007c1023e0 2023-10-08T09:55:32.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.854+0000 7f007bfff700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 0x7f00640a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f006c00b580 tx=0x7f006c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:32.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:32.858+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f007404f100 con 0x7f007c1023e0 2023-10-08T09:55:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.095+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f007c04e4c0 con 0x7f007c1023e0 2023-10-08T09:55:33.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.096+0000 7f00797fa700 1 -- 172.21.15.97:0/2533278618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2953 (secure 0 0 0) 0x7f0074019ba0 con 0x7f007c1023e0 2023-10-08T09:55:33.097 INFO:teuthology.orchestra.run.smithi097.stdout:e24 2023-10-08T09:55:33.097 INFO:teuthology.orchestra.run.smithi097.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:55:33.097 INFO:teuthology.orchestra.run.smithi097.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:55:33.097 INFO:teuthology.orchestra.run.smithi097.stdout:legacy client fscid: 1 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem 'cephfs' (1) 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:fs_name cephfs 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 24 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:created 2023-10-08T09:53:56.599415+0000 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2023-10-08T09:54:24.185261+0000 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:tableserver 0 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:root 0 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:session_timeout 300 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:session_autoclose 300 2023-10-08T09:55:33.098 INFO:teuthology.orchestra.run.smithi097.stdout:max_file_size 1099511627776 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:max_xattr_size 65536 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:required_client_features {} 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure 0 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure_osd_epoch 0 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:max_mds 3 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:in 0,1,2 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:up {0=24566,1=24562,2=24541} 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:failed 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:damaged 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:stopped 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:data_pools [3,4] 2023-10-08T09:55:33.099 INFO:teuthology.orchestra.run.smithi097.stdout:metadata_pool 2 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:inline_data disabled 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:balancer 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:bal_rank_mask -1 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:standby_count_wanted 0 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.h{0:24536} state up:standby-replay seq 1 addr [v2:172.21.15.178:6836/2015632800,v1:172.21.15.178:6837/2015632800] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.i{1:24562} state up:active seq 11 addr [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.f{1:24532} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/3587378302,v1:172.21.15.186:6835/3587378302] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c{2:24541} state up:active seq 12 addr [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.e{2:24518} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/2717102390,v1:172.21.15.178:6835/2717102390] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:33.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:Standby daemons: 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.j{-1:14685} state up:standby seq 1 addr [v2:172.21.15.97:6834/2090847706,v1:172.21.15.97:6835/2090847706] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d{-1:14697} state up:standby seq 1 addr [v2:172.21.15.97:6836/3214411301,v1:172.21.15.97:6837/3214411301] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.g{-1:14754} state up:standby seq 1 addr [v2:172.21.15.97:6838/1211581786,v1:172.21.15.97:6839/1211581786] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a{-1:14766} state up:standby seq 1 addr [v2:172.21.15.97:6840/3143185366,v1:172.21.15.97:6841/3143185366] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.k{-1:24500} state up:standby seq 1 addr [v2:172.21.15.178:6832/1070195466,v1:172.21.15.178:6833/1070195466] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.101 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.l{-1:24502} state up:standby seq 1 addr [v2:172.21.15.186:6832/3404847957,v1:172.21.15.186:6833/3404847957] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.097+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 msgr2=0x7f00640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.097+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 0x7f00640a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f006c00b580 tx=0x7f006c005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 msgr2=0x7f007c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f007400b9a0 tx=0x7f007400bcb0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 shutdown_connections 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f00640a1480 0x7f00640a3940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c1023e0 0x7f007c19dd20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f007c100840 0x7f007c1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 --2- 172.21.15.97:0/2533278618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f007c101a40 0x7f007c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 >> 172.21.15.97:0/2533278618 conn(0x7f007c0fbe00 msgr2=0x7f007c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 shutdown_connections 2023-10-08T09:55:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:33.098+0000 7f00825ea700 1 -- 172.21.15.97:0/2533278618 wait complete. 2023-10-08T09:55:33.103 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:33.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:33 smithi097 ceph-mon[99491]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:33.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:33 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2533278618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:55:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:33 smithi186 ceph-mon[104393]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:33 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2533278618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:55:33.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-08T09:55:33.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:33 smithi178 ceph-mon[103156]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:33.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:33 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2533278618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:55:34.113 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 -- 172.21.15.97:0/2798270924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ff730 msgr2=0x7f764c0ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 --2- 172.21.15.97:0/2798270924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ff730 0x7f764c0ffb50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f763c0099c0 tx=0x7f763c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 -- 172.21.15.97:0/2798270924 shutdown_connections 2023-10-08T09:55:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 --2- 172.21.15.97:0/2798270924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 --2- 172.21.15.97:0/2798270924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c0ff730 0x7f764c0ffb50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 --2- 172.21.15.97:0/2798270924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c100930 0x7f764c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 -- 172.21.15.97:0/2798270924 >> 172.21.15.97:0/2798270924 conn(0x7f764c0facf0 msgr2=0x7f764c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 -- 172.21.15.97:0/2798270924 shutdown_connections 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.338+0000 7f76526c2700 1 -- 172.21.15.97:0/2798270924 wait complete. 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 Processor -- start 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 -- start start 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c0ff730 0x7f764c103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1029f0 con 0x7f764c100930 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c102b60 con 0x7f764c0ff730 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f76526c2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1a0870 con 0x7f764c1012d0 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56488/0 (socket says 172.21.15.97:56488) 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 -- 172.21.15.97:0/3927266737 learned_addr learned my addr 172.21.15.97:0/3927266737 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764bfff700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c0ff730 0x7f764c103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f7650c5f700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 -- 172.21.15.97:0/3927266737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 msgr2=0x7f764c102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 -- 172.21.15.97:0/3927266737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c0ff730 msgr2=0x7f764c103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c0ff730 0x7f764c103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 -- 172.21.15.97:0/3927266737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0095f0 con 0x7f764c100930 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f7650c5f700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.339+0000 7f764b7fe700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f764000b530 tx=0x7f764000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.340+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640011830 con 0x7f764c100930 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.340+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7640011e50 con 0x7f764c100930 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.340+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000f2e0 con 0x7f764c100930 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.340+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c1a0b00 con 0x7f764c100930 2023-10-08T09:55:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.340+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1a1000 con 0x7f764c100930 2023-10-08T09:55:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.341+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c04e4c0 con 0x7f764c100930 2023-10-08T09:55:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.342+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7640010580 con 0x7f764c100930 2023-10-08T09:55:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.342+0000 7f76497fa700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 0x7f76340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.342+0000 7f764bfff700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 0x7f76340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.343+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76400ccac0 con 0x7f764c100930 2023-10-08T09:55:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.343+0000 7f764bfff700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 0x7f76340a39c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b580 tx=0x7f763c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.347+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7640095e10 con 0x7f764c100930 2023-10-08T09:55:35.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:35 smithi097 ceph-mon[99491]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:35 smithi186 ceph-mon[104393]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:35.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.538+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f764c10afb0 con 0x7f764c100930 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.539+0000 7f76497fa700 1 -- 172.21.15.97:0/3927266737 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5605 (secure 0 0 0) 0x7f7640099430 con 0x7f764c100930 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 87 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:created 2023-10-08T09:46:49.063848+0000 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2023-10-08T09:54:47.232892+0000 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 27 2023-10-08T09:55:35.540 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client luminous 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release reef 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-08T09:55:35.541 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 12 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6802/3104991007,v1:172.21.15.97:6803/3104991007] [v2:172.21.15.97:6804/3104991007,v1:172.21.15.97:6805/3104991007] exists,up 31b0d724-2bfc-4c58-8ac5-38962f5d300a 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/4284206053,v1:172.21.15.178:6801/4284206053] [v2:172.21.15.178:6802/4284206053,v1:172.21.15.178:6803/4284206053] exists,up 946de8df-41b4-46e5-87da-4822fa096d12 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/1978590862,v1:172.21.15.186:6801/1978590862] [v2:172.21.15.186:6802/1978590862,v1:172.21.15.186:6803/1978590862] exists,up 7dcdc682-9882-47fb-889a-4d412fbc7433 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6810/105487765,v1:172.21.15.97:6811/105487765] [v2:172.21.15.97:6812/105487765,v1:172.21.15.97:6813/105487765] exists,up 1a30f31c-10a5-4538-8fd9-35acf14be3eb 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 32 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/1866336071,v1:172.21.15.178:6809/1866336071] [v2:172.21.15.178:6810/1866336071,v1:172.21.15.178:6811/1866336071] exists,up bd3cbf7e-d161-4d7f-959e-5943a5a63bd6 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/2325993669,v1:172.21.15.186:6809/2325993669] [v2:172.21.15.186:6810/2325993669,v1:172.21.15.186:6811/2325993669] exists,up 7091360b-fa15-4e88-a5cc-44dcebef03c0 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 44 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6818/3267640836,v1:172.21.15.97:6819/3267640836] [v2:172.21.15.97:6820/3267640836,v1:172.21.15.97:6821/3267640836] exists,up facc86e5-e587-42d9-bbae-6f98916a084d 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/2550553059,v1:172.21.15.178:6817/2550553059] [v2:172.21.15.178:6818/2550553059,v1:172.21.15.178:6819/2550553059] exists,up 9764524a-05a3-4d06-83a8-c9c4ba6ec38c 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 up in weight 1 up_from 56 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/426737999,v1:172.21.15.186:6817/426737999] [v2:172.21.15.186:6818/426737999,v1:172.21.15.186:6819/426737999] exists,up f4415829-318a-4bb8-8e77-feda54434e5f 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 up in weight 1 up_from 63 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6826/3522179908,v1:172.21.15.97:6827/3522179908] [v2:172.21.15.97:6828/3522179908,v1:172.21.15.97:6829/3522179908] exists,up 0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91 2023-10-08T09:55:35.542 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 up in weight 1 up_from 68 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/3571291491,v1:172.21.15.178:6825/3571291491] [v2:172.21.15.178:6826/3571291491,v1:172.21.15.178:6827/3571291491] exists,up 9b0e6cc7-619d-466b-9ede-4bfc9d404bba 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 up in weight 1 up_from 76 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/1987590926,v1:172.21.15.186:6825/1987590926] [v2:172.21.15.186:6826/1987590926,v1:172.21.15.186:6827/1987590926] exists,up 34036837-0a5c-4d09-b8cb-ff074305f8b5 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.5 [8,11] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.f [8,11] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.10 [2,1] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.14 [2,4] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.1 [4,9] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.36 [4,9] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 4.2 [9,1] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 4.f [9,8] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 4.15 [5,10] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 4.19 [9,8] 2023-10-08T09:55:35.543 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/4256028615 expires 2023-10-09T09:47:45.797606+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/3827116852 expires 2023-10-09T09:47:22.724740+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/1873332931 expires 2023-10-09T09:47:45.797606+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/1217300114 expires 2023-10-09T09:47:22.724740+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/4047632094 expires 2023-10-09T09:47:45.797606+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/3827116852 expires 2023-10-09T09:47:22.724740+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/1873332931 expires 2023-10-09T09:47:45.797606+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/353854862 expires 2023-10-09T09:47:22.724740+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/2928584164 expires 2023-10-09T09:47:22.724740+0000 2023-10-08T09:55:35.544 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3484524433 expires 2023-10-09T09:47:45.797606+0000 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 msgr2=0x7f76340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 0x7f76340a39c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b580 tx=0x7f763c005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 msgr2=0x7f764c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f764000b530 tx=0x7f764000b8f0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 shutdown_connections 2023-10-08T09:55:35.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76340a1500 0x7f76340a39c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f764c1012d0 0x7f764c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c0ff730 0x7f764c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 --2- 172.21.15.97:0/3927266737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f764c100930 0x7f764c101d10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 >> 172.21.15.97:0/3927266737 conn(0x7f764c0facf0 msgr2=0x7f764c0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 shutdown_connections 2023-10-08T09:55:35.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:35.541+0000 7f76526c2700 1 -- 172.21.15.97:0/3927266737 wait complete. 2023-10-08T09:55:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:35 smithi178 ceph-mon[103156]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:35.964 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-08T09:55:35.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 -- 172.21.15.97:0/1302287607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc0ff5a0 msgr2=0x7f8cbc0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 --2- 172.21.15.97:0/1302287607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc0ff9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0099c0 tx=0x7f8cac009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 -- 172.21.15.97:0/1302287607 shutdown_connections 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 --2- 172.21.15.97:0/1302287607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc101130 0x7f8cbc1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 --2- 172.21.15.97:0/1302287607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc0ff9c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 --2- 172.21.15.97:0/1302287607 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc100790 0x7f8cbc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 -- 172.21.15.97:0/1302287607 >> 172.21.15.97:0/1302287607 conn(0x7f8cbc0fab10 msgr2=0x7f8cbc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 -- 172.21.15.97:0/1302287607 shutdown_connections 2023-10-08T09:55:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.099+0000 7f8cc2411700 1 -- 172.21.15.97:0/1302287607 wait complete. 2023-10-08T09:55:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 Processor -- start 2023-10-08T09:55:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 -- start start 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101130 0x7f8cbc108a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1091d0 con 0x7f8cbc100790 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc109340 con 0x7f8cbc0ff5a0 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.100+0000 7f8cc2411700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1a06e0 con 0x7f8cbc101130 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56500/0 (socket says 172.21.15.97:56500) 2023-10-08T09:55:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 -- 172.21.15.97:0/739167909 learned_addr learned my addr 172.21.15.97:0/739167909 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cc09ae700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101130 0x7f8cbc108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbbfff700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 -- 172.21.15.97:0/739167909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101130 msgr2=0x7f8cbc108a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101130 0x7f8cbc108a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 -- 172.21.15.97:0/739167909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 msgr2=0x7f8cbc109e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc109e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 -- 172.21.15.97:0/739167909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0095f0 con 0x7f8cbc100790 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbbfff700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cbb7fe700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d950 tx=0x7f8ca400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4009ae0 con 0x7f8cbc100790 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca4009c40 con 0x7f8cbc100790 2023-10-08T09:55:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400f620 con 0x7f8cbc100790 2023-10-08T09:55:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc1a0970 con 0x7f8cbc100790 2023-10-08T09:55:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.101+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1a0e30 con 0x7f8cbc100790 2023-10-08T09:55:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.102+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc106a20 con 0x7f8cbc100790 2023-10-08T09:55:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.103+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ca4010460 con 0x7f8cbc100790 2023-10-08T09:55:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.104+0000 7f8cb97fa700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 0x7f8ca80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.104+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8ca40ccf60 con 0x7f8cbc100790 2023-10-08T09:55:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.104+0000 7f8cbbfff700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 0x7f8ca80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.104+0000 7f8cbbfff700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 0x7f8ca80a3940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.108+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ca40962b0 con 0x7f8cbc100790 2023-10-08T09:55:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.301+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cbc10bf60 con 0x7f8cbc100790 2023-10-08T09:55:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.302+0000 7f8cb97fa700 1 -- 172.21.15.97:0/739167909 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7f8ca4016090 con 0x7f8cbc100790 2023-10-08T09:55:36.303 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:36.304 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:36.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.304+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 msgr2=0x7f8ca80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.304+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 0x7f8ca80a3940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac009f10 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.304+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 msgr2=0x7f8cbc1084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.304+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400d950 tx=0x7f8ca400dd10 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 shutdown_connections 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8ca80a1480 0x7f8ca80a3940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8cbc101130 0x7f8cbc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cbc0ff5a0 0x7f8cbc109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 --2- 172.21.15.97:0/739167909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cbc100790 0x7f8cbc1084f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 >> 172.21.15.97:0/739167909 conn(0x7f8cbc0fab10 msgr2=0x7f8cbc103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 shutdown_connections 2023-10-08T09:55:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.305+0000 7f8cc2411700 1 -- 172.21.15.97:0/739167909 wait complete. 2023-10-08T09:55:36.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:36.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- 172.21.15.97:0/1219563319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e80fffa0 msgr2=0x7f83e8100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 --2- 172.21.15.97:0/1219563319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e80fffa0 0x7f83e8100420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f83d400aaa0 tx=0x7f83d400adb0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- 172.21.15.97:0/1219563319 shutdown_connections 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 --2- 172.21.15.97:0/1219563319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e8100960 0x7f83e8107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 --2- 172.21.15.97:0/1219563319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fee50 0x7f83e80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 --2- 172.21.15.97:0/1219563319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e80fffa0 0x7f83e8100420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- 172.21.15.97:0/1219563319 >> 172.21.15.97:0/1219563319 conn(0x7f83e80fa3c0 msgr2=0x7f83e80fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- 172.21.15.97:0/1219563319 shutdown_connections 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- 172.21.15.97:0/1219563319 wait complete. 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 Processor -- start 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.444+0000 7f83eebd5700 1 -- start start 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 0x7f83e8194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fffa0 0x7f83e8194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e819c060 con 0x7f83e8100960 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e819c1a0 con 0x7f83e80fffa0 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e819c330 con 0x7f83e80fee50 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56516/0 (socket says 172.21.15.97:56516) 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 -- 172.21.15.97:0/3843547323 learned_addr learned my addr 172.21.15.97:0/3843547323 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ec971700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 0x7f83e8194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83e7fff700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fffa0 0x7f83e8194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 -- 172.21.15.97:0/3843547323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 msgr2=0x7f83e8194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 0x7f83e8194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 -- 172.21.15.97:0/3843547323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fffa0 msgr2=0x7f83e8194a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fffa0 0x7f83e8194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 -- 172.21.15.97:0/3843547323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d400a700 con 0x7f83e8100960 2023-10-08T09:55:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ec971700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 0x7f83e8194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83ed172700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f83e0007ee0 tx=0x7f83e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0017070 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83e000f040 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e819c610 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.445+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0005740 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.446+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e819cad0 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.447+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e8106210 con 0x7f83e8100960 2023-10-08T09:55:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.448+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f83e00058a0 con 0x7f83e8100960 2023-10-08T09:55:36.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.448+0000 7f83e5ffb700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 0x7f83d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.448+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83e00cc8e0 con 0x7f83e8100960 2023-10-08T09:55:36.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.448+0000 7f83ec971700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 0x7f83d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.449+0000 7f83ec971700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 0x7f83d80a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005fd0 tx=0x7f83dc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.453+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f83e0095c30 con 0x7f83e8100960 2023-10-08T09:55:36.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:36 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3927266737' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:55:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.643+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83e8105bf0 con 0x7f83e8100960 2023-10-08T09:55:36.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.643+0000 7f83e5ffb700 1 -- 172.21.15.97:0/3843547323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83e0099250 con 0x7f83e8100960 2023-10-08T09:55:36.644 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 msgr2=0x7f83d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 0x7f83d80a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005fd0 tx=0x7f83dc005ee0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 msgr2=0x7f83e819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f83e0007ee0 tx=0x7f83e000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 shutdown_connections 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f83d80a1480 0x7f83d80a3940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83e8100960 0x7f83e819ba90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e80fee50 0x7f83e8194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 --2- 172.21.15.97:0/3843547323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83e80fffa0 0x7f83e8194a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 >> 172.21.15.97:0/3843547323 conn(0x7f83e80fa3c0 msgr2=0x7f83e8103350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 shutdown_connections 2023-10-08T09:55:36.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.645+0000 7f83eebd5700 1 -- 172.21.15.97:0/3843547323 wait complete. 2023-10-08T09:55:36.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:36 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3927266737' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:55:36.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:36.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:36 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3927266737' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:55:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/1946962422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c040ff6b0 msgr2=0x7f2c040ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/1946962422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c040ff6b0 0x7f2c040ffad0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2c000099c0 tx=0x7f2c00009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/1946962422 shutdown_connections 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/1946962422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c04107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/1946962422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c040ff6b0 0x7f2c040ffad0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/1946962422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c041008a0 0x7f2c04100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/1946962422 >> 172.21.15.97:0/1946962422 conn(0x7f2c040fac40 msgr2=0x7f2c040fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/1946962422 shutdown_connections 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/1946962422 wait complete. 2023-10-08T09:55:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 Processor -- start 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.782+0000 7f2c0c1f1700 1 -- start start 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c040ff6b0 0x7f2c040695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c0419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0419e170 con 0x7f2c041008a0 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0419e2e0 con 0x7f2c04101240 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0419e4a0 con 0x7f2c040ff6b0 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56528/0 (socket says 172.21.15.97:56528) 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 -- 172.21.15.97:0/2544500423 learned_addr learned my addr 172.21.15.97:0/2544500423 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:36.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c09f8d700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c040ff6b0 0x7f2c040695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0a78e700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c0419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 -- 172.21.15.97:0/2544500423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c040ff6b0 msgr2=0x7f2c040695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c040ff6b0 0x7f2c040695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 -- 172.21.15.97:0/2544500423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 msgr2=0x7f2c0419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c0419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 -- 172.21.15.97:0/2544500423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000095f0 con 0x7f2c041008a0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0a78e700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c0419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0978c700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c9b0 tx=0x7f2bf400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007b00 con 0x7f2c041008a0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf4007c60 con 0x7f2c041008a0 2023-10-08T09:55:36.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0419e780 con 0x7f2c041008a0 2023-10-08T09:55:36.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4018660 con 0x7f2c041008a0 2023-10-08T09:55:36.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.783+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0419ec90 con 0x7f2c041008a0 2023-10-08T09:55:36.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.784+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c04106ae0 con 0x7f2c041008a0 2023-10-08T09:55:36.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.785+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2bf400f450 con 0x7f2c041008a0 2023-10-08T09:55:36.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.786+0000 7f2bfaffd700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 0x7f2bf00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:36.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.786+0000 7f2c09f8d700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 0x7f2bf00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:36.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.786+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2bf40cd2b0 con 0x7f2c041008a0 2023-10-08T09:55:36.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.786+0000 7f2c09f8d700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 0x7f2bf00a3940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006010 tx=0x7f2c00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:36.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.789+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2bf4096690 con 0x7f2c041008a0 2023-10-08T09:55:36.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.980+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c0410afe0 con 0x7f2c041008a0 2023-10-08T09:55:36.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.981+0000 7f2bfaffd700 1 -- 172.21.15.97:0/2544500423 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bf4099cb0 con 0x7f2c041008a0 2023-10-08T09:55:36.982 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 msgr2=0x7f2bf00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 0x7f2bf00a3940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006010 tx=0x7f2c00000bc0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 msgr2=0x7f2c0419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c9b0 tx=0x7f2bf400ccc0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 shutdown_connections 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2bf00a1480 0x7f2bf00a3940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c04101240 0x7f2c0419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c040ff6b0 0x7f2c040695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 --2- 172.21.15.97:0/2544500423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c041008a0 0x7f2c0419b380 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 >> 172.21.15.97:0/2544500423 conn(0x7f2c040fac40 msgr2=0x7f2c04103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 shutdown_connections 2023-10-08T09:55:36.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:36.983+0000 7f2c0c1f1700 1 -- 172.21.15.97:0/2544500423 wait complete. 2023-10-08T09:55:36.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 -- 172.21.15.97:0/1069335543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc581002f0 msgr2=0x7efc58107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/1069335543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc581002f0 0x7efc58107b90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b0a0 tx=0x7efc5000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 -- 172.21.15.97:0/1069335543 shutdown_connections 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/1069335543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc581002f0 0x7efc58107b90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/1069335543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc58100ce0 0x7efc58101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/1069335543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc580ff930 0x7efc580ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.119+0000 7efc5cbd6700 1 -- 172.21.15.97:0/1069335543 >> 172.21.15.97:0/1069335543 conn(0x7efc580fa3c0 msgr2=0x7efc580fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 -- 172.21.15.97:0/1069335543 shutdown_connections 2023-10-08T09:55:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 -- 172.21.15.97:0/1069335543 wait complete. 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 Processor -- start 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 -- start start 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc580ff930 0x7efc580ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc581002f0 0x7efc5819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5819e280 con 0x7efc58100ce0 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.120+0000 7efc5cbd6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5819e3f0 con 0x7efc580ff930 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc5cbd6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5819e5b0 con 0x7efc581002f0 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc5659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc580ff930 0x7efc580ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56544/0 (socket says 172.21.15.97:56544) 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 -- 172.21.15.97:0/363002001 learned_addr learned my addr 172.21.15.97:0/363002001 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc55d9b700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc581002f0 0x7efc5819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 -- 172.21.15.97:0/363002001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc581002f0 msgr2=0x7efc5819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc581002f0 0x7efc5819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 -- 172.21.15.97:0/363002001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc580ff930 msgr2=0x7efc580ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc580ff930 0x7efc580ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 -- 172.21.15.97:0/363002001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009d00 con 0x7efc58100ce0 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc56d9d700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b830 tx=0x7efc50004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000e040 con 0x7efc58100ce0 2023-10-08T09:55:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5819e830 con 0x7efc58100ce0 2023-10-08T09:55:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc50004e90 con 0x7efc58100ce0 2023-10-08T09:55:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5001bb80 con 0x7efc58100ce0 2023-10-08T09:55:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.121+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5819ecb0 con 0x7efc58100ce0 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.122+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc58106aa0 con 0x7efc58100ce0 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.123+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efc5001bce0 con 0x7efc58100ce0 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.124+0000 7efc477fe700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 0x7efc400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.124+0000 7efc5659c700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 0x7efc400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.124+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc500cf140 con 0x7efc58100ce0 2023-10-08T09:55:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.125+0000 7efc5659c700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 0x7efc400a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efc58068ab0 tx=0x7efc4800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.128+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efc50098520 con 0x7efc58100ce0 2023-10-08T09:55:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.322+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc5810b0b0 con 0x7efc58100ce0 2023-10-08T09:55:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.322+0000 7efc477fe700 1 -- 172.21.15.97:0/363002001 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc5009bb40 con 0x7efc58100ce0 2023-10-08T09:55:37.323 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.324+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 msgr2=0x7efc400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.324+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 0x7efc400a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efc58068ab0 tx=0x7efc4800a3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.324+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 msgr2=0x7efc5819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.324+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b830 tx=0x7efc50004a50 comp rx=0 tx=0).stop 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.324+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 shutdown_connections 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efc400a1480 0x7efc400a3940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc58100ce0 0x7efc5819dae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc580ff930 0x7efc580ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 --2- 172.21.15.97:0/363002001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc581002f0 0x7efc5819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 >> 172.21.15.97:0/363002001 conn(0x7efc580fa3c0 msgr2=0x7efc58103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 shutdown_connections 2023-10-08T09:55:37.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.325+0000 7efc5cbd6700 1 -- 172.21.15.97:0/363002001 wait complete. 2023-10-08T09:55:37.336 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:37 smithi097 ceph-mon[99491]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:37.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/739167909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:37.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3843547323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:37 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2544500423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 -- 172.21.15.97:0/4040848993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50101230 msgr2=0x7f8d50107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 --2- 172.21.15.97:0/4040848993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50101230 0x7f8d50107ae0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b0d0 tx=0x7f8d4800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 -- 172.21.15.97:0/4040848993 shutdown_connections 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 --2- 172.21.15.97:0/4040848993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50101230 0x7f8d50107ae0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 --2- 172.21.15.97:0/4040848993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d500ff6a0 0x7f8d500ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 --2- 172.21.15.97:0/4040848993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50100890 0x7f8d50100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 -- 172.21.15.97:0/4040848993 >> 172.21.15.97:0/4040848993 conn(0x7f8d500fac10 msgr2=0x7f8d500fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:37.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 -- 172.21.15.97:0/4040848993 shutdown_connections 2023-10-08T09:55:37.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 -- 172.21.15.97:0/4040848993 wait complete. 2023-10-08T09:55:37.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.461+0000 7f8d551fc700 1 Processor -- start 2023-10-08T09:55:37.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 -- start start 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d50100890 0x7f8d5019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50101230 0x7f8d5019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019e2e0 con 0x7f8d500ff6a0 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019e450 con 0x7f8d50100890 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019e610 con 0x7f8d50101230 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56552/0 (socket says 172.21.15.97:56552) 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 -- 172.21.15.97:0/3203707035 learned_addr learned my addr 172.21.15.97:0/3203707035 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:37.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4e59c700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d50100890 0x7f8d5019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4f59e700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50101230 0x7f8d5019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 -- 172.21.15.97:0/3203707035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50101230 msgr2=0x7f8d5019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50101230 0x7f8d5019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 -- 172.21.15.97:0/3203707035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d50100890 msgr2=0x7f8d5019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d50100890 0x7f8d5019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 -- 172.21.15.97:0/3203707035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009d00 con 0x7f8d500ff6a0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d4ed9d700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c990 tx=0x7f8d4400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44007b00 con 0x7f8d500ff6a0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.462+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5019e890 con 0x7f8d500ff6a0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.463+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d44007c60 con 0x7f8d500ff6a0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.463+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400fea0 con 0x7f8d500ff6a0 2023-10-08T09:55:37.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.463+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5019edf0 con 0x7f8d500ff6a0 2023-10-08T09:55:37.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.464+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d501069f0 con 0x7f8d500ff6a0 2023-10-08T09:55:37.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.465+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8d4400f450 con 0x7f8d500ff6a0 2023-10-08T09:55:37.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.465+0000 7f8d37fff700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 0x7f8d380a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.465+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d440ccf50 con 0x7f8d500ff6a0 2023-10-08T09:55:37.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.465+0000 7f8d4e59c700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 0x7f8d380a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.466+0000 7f8d4e59c700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 0x7f8d380a3940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d50068910 tx=0x7f8d4000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.469+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8d440962a0 con 0x7f8d500ff6a0 2023-10-08T09:55:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:37 smithi186 ceph-mon[104393]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:37 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/739167909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:37 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3843547323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:37 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2544500423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.658+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8d50067530 con 0x7f8d500ff6a0 2023-10-08T09:55:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.658+0000 7f8d37fff700 1 -- 172.21.15.97:0/3203707035 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8d50067530 con 0x7f8d500ff6a0 2023-10-08T09:55:37.659 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.660+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 msgr2=0x7f8d380a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.660+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 0x7f8d380a3940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d50068910 tx=0x7f8d4000a300 comp rx=0 tx=0).stop 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 msgr2=0x7f8d50068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c990 tx=0x7f8d4400cca0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 shutdown_connections 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8d380a1480 0x7f8d380a3940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d50101230 0x7f8d5019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500ff6a0 0x7f8d50068370 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 --2- 172.21.15.97:0/3203707035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d50100890 0x7f8d5019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 >> 172.21.15.97:0/3203707035 conn(0x7f8d500fac10 msgr2=0x7f8d50103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 shutdown_connections 2023-10-08T09:55:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.661+0000 7f8d551fc700 1 -- 172.21.15.97:0/3203707035 wait complete. 2023-10-08T09:55:37.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:37 smithi178 ceph-mon[103156]: pgmap v312: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:37 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/739167909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:37 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3843547323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:37 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2544500423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.793+0000 7f7190646700 1 -- 172.21.15.97:0/4026342582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71881007c0 msgr2=0x7f7188100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.793+0000 7f7190646700 1 --2- 172.21.15.97:0/4026342582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71881007c0 0x7f7188100c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71840099c0 tx=0x7f7184009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- 172.21.15.97:0/4026342582 shutdown_connections 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- 172.21.15.97:0/4026342582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7188101160 0x7f7188107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- 172.21.15.97:0/4026342582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71880ff5d0 0x7f71880ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- 172.21.15.97:0/4026342582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71881007c0 0x7f7188100c20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- 172.21.15.97:0/4026342582 >> 172.21.15.97:0/4026342582 conn(0x7f71880fab40 msgr2=0x7f71880fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- 172.21.15.97:0/4026342582 shutdown_connections 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- 172.21.15.97:0/4026342582 wait complete. 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 Processor -- start 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- start start 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880ff5d0 0x7f71881955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71881007c0 0x7f7188195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718819d0e0 con 0x7f7188101160 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718819d220 con 0x7f71881007c0 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f7190646700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718819d3b0 con 0x7f71880ff5d0 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.794+0000 7f718ebe3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56570/0 (socket says 172.21.15.97:56570) 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 -- 172.21.15.97:0/2201133211 learned_addr learned my addr 172.21.15.97:0/2201133211 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718dbe1700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71881007c0 0x7f7188195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718e3e2700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880ff5d0 0x7f71881955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 -- 172.21.15.97:0/2201133211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880ff5d0 msgr2=0x7f71881955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880ff5d0 0x7f71881955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 -- 172.21.15.97:0/2201133211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71881007c0 msgr2=0x7f7188195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71881007c0 0x7f7188195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 -- 172.21.15.97:0/2201133211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71840095f0 con 0x7f7188101160 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f718ebe3700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f718000b970 tx=0x7f718000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000d860 con 0x7f7188101160 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718000de80 con 0x7f7188101160 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180017400 con 0x7f7188101160 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718819d690 con 0x7f7188101160 2023-10-08T09:55:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.795+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718819dba0 con 0x7f7188101160 2023-10-08T09:55:37.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.796+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7188106a00 con 0x7f7188101160 2023-10-08T09:55:37.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.797+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f718000e630 con 0x7f7188101160 2023-10-08T09:55:37.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.798+0000 7f717f7fe700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 0x7f71740a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:37.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.798+0000 7f718e3e2700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 0x7f71740a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.798+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7180006e40 con 0x7f7188101160 2023-10-08T09:55:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.798+0000 7f718e3e2700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 0x7f71740a3a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7188100620 tx=0x7f7178009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:37.801+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7180096110 con 0x7f7188101160 2023-10-08T09:55:38.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.036+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7188066bc0 con 0x7f7188101160 2023-10-08T09:55:38.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.037+0000 7f717f7fe700 1 -- 172.21.15.97:0/2201133211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f7180099730 con 0x7f7188101160 2023-10-08T09:55:38.038 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:38.038 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 msgr2=0x7f71740a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 0x7f71740a3a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7188100620 tx=0x7f7178009260 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 msgr2=0x7f718819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f718000b970 tx=0x7f718000bc80 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 shutdown_connections 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71740a15d0 0x7f71740a3a90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7188101160 0x7f718819cb10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71880ff5d0 0x7f71881955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 --2- 172.21.15.97:0/2201133211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71881007c0 0x7f7188195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 >> 172.21.15.97:0/2201133211 conn(0x7f71880fab40 msgr2=0x7f7188103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 shutdown_connections 2023-10-08T09:55:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.039+0000 7f7190646700 1 -- 172.21.15.97:0/2201133211 wait complete. 2023-10-08T09:55:38.041 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:38.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 -- 172.21.15.97:0/2937886501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0101cb0 msgr2=0x7fcbb01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/2937886501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0101cb0 0x7fcbb01020d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009990 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 -- 172.21.15.97:0/2937886501 shutdown_connections 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/2937886501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/2937886501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb0101cb0 0x7fcbb01020d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/2937886501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb00feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 -- 172.21.15.97:0/2937886501 >> 172.21.15.97:0/2937886501 conn(0x7fcbb00fa2c0 msgr2=0x7fcbb00fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 -- 172.21.15.97:0/2937886501 shutdown_connections 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.173+0000 7fcbb72a7700 1 -- 172.21.15.97:0/2937886501 wait complete. 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 Processor -- start 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 -- start start 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb0196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101cb0 0x7fcbb019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019e160 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019e2a0 con 0x7fcbb00fe720 2023-10-08T09:55:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb72a7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019e430 con 0x7fcbb0101cb0 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56580/0 (socket says 172.21.15.97:56580) 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 -- 172.21.15.97:0/3885704352 learned_addr learned my addr 172.21.15.97:0/3885704352 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb5043700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb0196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 -- 172.21.15.97:0/3885704352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101cb0 msgr2=0x7fcbb019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101cb0 0x7fcbb019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 -- 172.21.15.97:0/3885704352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 msgr2=0x7fcbb0196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb0196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 -- 172.21.15.97:0/3885704352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00095f0 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb5043700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb0196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.174+0000 7fcbb4842700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c970 tx=0x7fcbac00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.175+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac004220 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.175+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac007d60 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.175+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb019e710 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.175+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac003d00 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.175+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb019ec20 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.176+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0107cd0 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.177+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcbac004380 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.177+0000 7fcba67fc700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 0x7fcb9c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.177+0000 7fcbb5043700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 0x7fcb9c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.177+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcbac0ccf70 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.178+0000 7fcbb5043700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 0x7fcb9c0a3940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba00037f0 tx=0x7fcba0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.181+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcbac0962c0 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.375+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbb010bec0 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.377+0000 7fcba67fc700 1 -- 172.21.15.97:0/3885704352 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7fcbac01d090 con 0x7fcbb00ff0e0 2023-10-08T09:55:38.378 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:38.378 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 msgr2=0x7fcb9c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 0x7fcb9c0a3940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba00037f0 tx=0x7fcba0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 msgr2=0x7fcbb0196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c970 tx=0x7fcbac00cd30 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 shutdown_connections 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fcb9c0a1480 0x7fcb9c0a3940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101cb0 0x7fcbb019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00fe720 0x7fcbb0196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 --2- 172.21.15.97:0/3885704352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb00ff0e0 0x7fcbb0196b70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 >> 172.21.15.97:0/3885704352 conn(0x7fcbb00fa2c0 msgr2=0x7fcbb0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 shutdown_connections 2023-10-08T09:55:38.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.379+0000 7fcbb72a7700 1 -- 172.21.15.97:0/3885704352 wait complete. 2023-10-08T09:55:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/363002001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3203707035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.391 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:38 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2201133211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:38.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:38.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 -- 172.21.15.97:0/3123934066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 msgr2=0x7f9fe0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/3123934066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0100be0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00099c0 tx=0x7f9fd0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 -- 172.21.15.97:0/3123934066 shutdown_connections 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/3123934066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe0102350 0x7f9fe0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/3123934066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0100be0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/3123934066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 0x7f9fe0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 -- 172.21.15.97:0/3123934066 >> 172.21.15.97:0/3123934066 conn(0x7f9fe00fbd30 msgr2=0x7f9fe00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 -- 172.21.15.97:0/3123934066 shutdown_connections 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.516+0000 7f9fe83cb700 1 -- 172.21.15.97:0/3123934066 wait complete. 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 Processor -- start 2023-10-08T09:55:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 -- start start 2023-10-08T09:55:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 0x7f9fe0196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe0102350 0x7f9fe019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019e290 con 0x7f9fe01007c0 2023-10-08T09:55:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019e3d0 con 0x7f9fe0102350 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe83cb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019e560 con 0x7f9fe01019b0 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56592/0 (socket says 172.21.15.97:56592) 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 -- 172.21.15.97:0/750071293 learned_addr learned my addr 172.21.15.97:0/750071293 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe5966700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 0x7f9fe0196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 -- 172.21.15.97:0/750071293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 msgr2=0x7f9fe0196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 0x7f9fe0196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 -- 172.21.15.97:0/750071293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe0102350 msgr2=0x7f9fe019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe0102350 0x7f9fe019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 -- 172.21.15.97:0/750071293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00095f0 con 0x7f9fe01007c0 2023-10-08T09:55:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.517+0000 7f9fe6167700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0003820 tx=0x7f9fd0005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.518+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd001d070 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.518+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe019e7e0 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.518+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fd0004110 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.518+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0022e40 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.518+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe019ec90 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.519+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe0107c90 con 0x7f9fe01007c0 2023-10-08T09:55:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.520+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9fd0022570 con 0x7f9fe01007c0 2023-10-08T09:55:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.520+0000 7f9fd77fe700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 0x7f9fcc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.520+0000 7f9fe5966700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 0x7f9fcc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.520+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9fd00ce300 con 0x7f9fe01007c0 2023-10-08T09:55:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.521+0000 7f9fe5966700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 0x7f9fcc0a3940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005950 tx=0x7f9fdc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/363002001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3203707035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:38 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2201133211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.524+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fd0097780 con 0x7f9fe01007c0 2023-10-08T09:55:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/363002001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3203707035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:38 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2201133211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.715+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fe010bfe0 con 0x7f9fe01007c0 2023-10-08T09:55:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.716+0000 7f9fd77fe700 1 -- 172.21.15.97:0/750071293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fd0027020 con 0x7f9fe01007c0 2023-10-08T09:55:38.717 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 msgr2=0x7f9fcc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 0x7f9fcc0a3940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005950 tx=0x7f9fdc0058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 msgr2=0x7f9fe0196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0003820 tx=0x7f9fd0005ad0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 shutdown_connections 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fcc0a1480 0x7f9fcc0a3940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe0102350 0x7f9fe019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fe01007c0 0x7f9fe0196760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 --2- 172.21.15.97:0/750071293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fe01019b0 0x7f9fe0196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 >> 172.21.15.97:0/750071293 conn(0x7f9fe00fbd30 msgr2=0x7f9fe0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 shutdown_connections 2023-10-08T09:55:38.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.718+0000 7f9fe83cb700 1 -- 172.21.15.97:0/750071293 wait complete. 2023-10-08T09:55:38.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 -- 172.21.15.97:0/1665855335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c1008a0 msgr2=0x7f101c100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 --2- 172.21.15.97:0/1665855335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c1008a0 0x7f101c100d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f10040099c0 tx=0x7f1004009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 -- 172.21.15.97:0/1665855335 shutdown_connections 2023-10-08T09:55:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 --2- 172.21.15.97:0/1665855335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c101240 0x7f101c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 --2- 172.21.15.97:0/1665855335 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c0ff6b0 0x7f101c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 --2- 172.21.15.97:0/1665855335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c1008a0 0x7f101c100d00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 -- 172.21.15.97:0/1665855335 >> 172.21.15.97:0/1665855335 conn(0x7f101c0fac40 msgr2=0x7f101c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 -- 172.21.15.97:0/1665855335 shutdown_connections 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 -- 172.21.15.97:0/1665855335 wait complete. 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.854+0000 7f10223b4700 1 Processor -- start 2023-10-08T09:55:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 -- start start 2023-10-08T09:55:38.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 0x7f101c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c101240 0x7f101c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c19e230 con 0x7f101c0ff6b0 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c19e3a0 con 0x7f101c1008a0 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f10223b4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c19e560 con 0x7f101c101240 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56606/0 (socket says 172.21.15.97:56606) 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 -- 172.21.15.97:0/3338296624 learned_addr learned my addr 172.21.15.97:0/3338296624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101b7fe700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 0x7f101c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 -- 172.21.15.97:0/3338296624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c101240 msgr2=0x7f101c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f1020951700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c101240 0x7f101c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c101240 0x7f101c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 -- 172.21.15.97:0/3338296624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 msgr2=0x7f101c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:38.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 0x7f101c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101bfff700 1 -- 172.21.15.97:0/3338296624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040095f0 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.855+0000 7f101b7fe700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 0x7f101c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f101bfff700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c970 tx=0x7f100c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c004220 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c007d60 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c19e7e0 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c003d00 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.856+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c19ecf0 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.857+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c106b50 con 0x7f101c0ff6b0 2023-10-08T09:55:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.858+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f100c004380 con 0x7f101c0ff6b0 2023-10-08T09:55:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.858+0000 7f10197fa700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 0x7f10080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.858+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f100c0cd090 con 0x7f101c0ff6b0 2023-10-08T09:55:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.859+0000 7f101b7fe700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 0x7f10080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.859+0000 7f101b7fe700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 0x7f10080a3940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f1004009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:38.862+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f100c0963e0 con 0x7f101c0ff6b0 2023-10-08T09:55:39.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.051+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f101c10aec0 con 0x7f101c0ff6b0 2023-10-08T09:55:39.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.051+0000 7f10197fa700 1 -- 172.21.15.97:0/3338296624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f100c01d020 con 0x7f101c0ff6b0 2023-10-08T09:55:39.052 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 msgr2=0x7f10080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 0x7f10080a3940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f1004009530 comp rx=0 tx=0).stop 2023-10-08T09:55:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 msgr2=0x7f101c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c970 tx=0x7f100c00cd30 comp rx=0 tx=0).stop 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 shutdown_connections 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f10080a1480 0x7f10080a3940 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f101c101240 0x7f101c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0ff6b0 0x7f101c0697d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 --2- 172.21.15.97:0/3338296624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f101c1008a0 0x7f101c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 >> 172.21.15.97:0/3338296624 conn(0x7f101c0fac40 msgr2=0x7f101c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 shutdown_connections 2023-10-08T09:55:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.053+0000 7f10223b4700 1 -- 172.21.15.97:0/3338296624 wait complete. 2023-10-08T09:55:39.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.189+0000 7f4ea4902700 1 -- 172.21.15.97:0/3401791829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00ff6a0 msgr2=0x7f4ea00ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.189+0000 7f4ea4902700 1 --2- 172.21.15.97:0/3401791829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea00ffac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099c0 tx=0x7f4e90009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- 172.21.15.97:0/3401791829 shutdown_connections 2023-10-08T09:55:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- 172.21.15.97:0/3401791829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0101230 0x7f4ea0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- 172.21.15.97:0/3401791829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea00ffac0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- 172.21.15.97:0/3401791829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea0100890 0x7f4ea0100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- 172.21.15.97:0/3401791829 >> 172.21.15.97:0/3401791829 conn(0x7f4ea00fac10 msgr2=0x7f4ea00fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- 172.21.15.97:0/3401791829 shutdown_connections 2023-10-08T09:55:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- 172.21.15.97:0/3401791829 wait complete. 2023-10-08T09:55:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 Processor -- start 2023-10-08T09:55:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- start start 2023-10-08T09:55:39.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea0068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 0x7f4ea019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea019e240 con 0x7f4ea0101230 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea019e3b0 con 0x7f4ea00ff6a0 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.190+0000 7f4ea4902700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea019e570 con 0x7f4ea0100890 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56624/0 (socket says 172.21.15.97:56624) 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 -- 172.21.15.97:0/1460971427 learned_addr learned my addr 172.21.15.97:0/1460971427 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9dd9b700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 0x7f4ea019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9e59c700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea0068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 -- 172.21.15.97:0/1460971427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 msgr2=0x7f4ea019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 0x7f4ea019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 -- 172.21.15.97:0/1460971427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 msgr2=0x7f4ea0068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea0068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 -- 172.21.15.97:0/1460971427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900095f0 con 0x7f4ea0101230 2023-10-08T09:55:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9e59c700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea0068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9dd9b700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 0x7f4ea019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e9ed9d700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007ee0 tx=0x7f4e9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800ddd0 con 0x7f4ea0101230 2023-10-08T09:55:39.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e9800f040 con 0x7f4ea0101230 2023-10-08T09:55:39.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea019e850 con 0x7f4ea0101230 2023-10-08T09:55:39.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98015400 con 0x7f4ea0101230 2023-10-08T09:55:39.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.191+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea019ed10 con 0x7f4ea0101230 2023-10-08T09:55:39.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.192+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea0106bc0 con 0x7f4ea0101230 2023-10-08T09:55:39.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.194+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4e98004000 con 0x7f4ea0101230 2023-10-08T09:55:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.194+0000 7f4e8f7fe700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 0x7f4e880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.195+0000 7f4e9e59c700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 0x7f4e880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.195+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e980cd1e0 con 0x7f4ea0101230 2023-10-08T09:55:39.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.195+0000 7f4e9e59c700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 0x7f4e880a3940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006010 tx=0x7f4e90009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.198+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e98096530 con 0x7f4ea0101230 2023-10-08T09:55:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.389+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ea010afc0 con 0x7f4ea0101230 2023-10-08T09:55:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.390+0000 7f4e8f7fe700 1 -- 172.21.15.97:0/1460971427 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e980230a0 con 0x7f4ea0101230 2023-10-08T09:55:39.391 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 msgr2=0x7f4e880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 0x7f4e880a3940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006010 tx=0x7f4e90009530 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 msgr2=0x7f4ea019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007ee0 tx=0x7f4e9800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 shutdown_connections 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e880a1480 0x7f4e880a3940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0101230 0x7f4ea019daa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff6a0 0x7f4ea0068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 --2- 172.21.15.97:0/1460971427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea0100890 0x7f4ea019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 >> 172.21.15.97:0/1460971427 conn(0x7f4ea00fac10 msgr2=0x7f4ea0103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 shutdown_connections 2023-10-08T09:55:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.392+0000 7f4ea4902700 1 -- 172.21.15.97:0/1460971427 wait complete. 2023-10-08T09:55:39.404 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:39 smithi097 ceph-mon[99491]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:39.404 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3885704352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:39.404 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/750071293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.405 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:39 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3338296624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:39 smithi186 ceph-mon[104393]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3885704352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/750071293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:39 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3338296624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3612541062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d680feab0 msgr2=0x7f3d680feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3612541062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d680feab0 0x7f3d680feed0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009990 tx=0x7f3d58009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3612541062 shutdown_connections 2023-10-08T09:55:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3612541062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d68108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3612541062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d680feab0 0x7f3d680feed0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3612541062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d68101b20 0x7f3d68101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3612541062 >> 172.21.15.97:0/3612541062 conn(0x7f3d680fa3c0 msgr2=0x7f3d680fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3612541062 shutdown_connections 2023-10-08T09:55:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.528+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3612541062 wait complete. 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 Processor -- start 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 -- start start 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d680feab0 0x7f3d68196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d6819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819e2a0 con 0x7f3d68101b20 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819e3e0 con 0x7f3d681024c0 2023-10-08T09:55:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d6d5fa700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819e570 con 0x7f3d680feab0 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d667fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d667fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56634/0 (socket says 172.21.15.97:56634) 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d667fc700 1 -- 172.21.15.97:0/3539601461 learned_addr learned my addr 172.21.15.97:0/3539601461 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.529+0000 7f3d66ffd700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d680feab0 0x7f3d68196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d677fe700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d6819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 -- 172.21.15.97:0/3539601461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d680feab0 msgr2=0x7f3d68196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d680feab0 0x7f3d68196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 -- 172.21.15.97:0/3539601461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 msgr2=0x7f3d6819dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d6819dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 -- 172.21.15.97:0/3539601461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580095f0 con 0x7f3d68101b20 2023-10-08T09:55:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d677fe700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d6819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d667fc700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c920 tx=0x7f3d5c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c007b00 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c00ce70 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6819e850 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0054e0 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.530+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6819ed60 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.531+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68107e00 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.533+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3d5c004750 con 0x7f3d68101b20 2023-10-08T09:55:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.533+0000 7f3d4ffff700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 0x7f3d500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.534+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d5c0ccfe0 con 0x7f3d68101b20 2023-10-08T09:55:39.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.534+0000 7f3d66ffd700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 0x7f3d500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.534+0000 7f3d66ffd700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 0x7f3d500a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800b5c0 tx=0x7f3d58003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.537+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d5c096310 con 0x7f3d68101b20 2023-10-08T09:55:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:39 smithi178 ceph-mon[103156]: pgmap v313: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-08T09:55:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3885704352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/750071293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:39 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3338296624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.727+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3d6810c0d0 con 0x7f3d68101b20 2023-10-08T09:55:39.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.728+0000 7f3d4ffff700 1 -- 172.21.15.97:0/3539601461 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3d5c01d020 con 0x7f3d68101b20 2023-10-08T09:55:39.729 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 msgr2=0x7f3d500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 0x7f3d500a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800b5c0 tx=0x7f3d58003790 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 msgr2=0x7f3d68196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00c920 tx=0x7f3d5c00cc30 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 shutdown_connections 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3d500a1480 0x7f3d500a3940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d681024c0 0x7f3d6819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d680feab0 0x7f3d68196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 --2- 172.21.15.97:0/3539601461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d68101b20 0x7f3d68196cb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 >> 172.21.15.97:0/3539601461 conn(0x7f3d680fa3c0 msgr2=0x7f3d68104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 shutdown_connections 2023-10-08T09:55:39.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.730+0000 7f3d6d5fa700 1 -- 172.21.15.97:0/3539601461 wait complete. 2023-10-08T09:55:39.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 -- 172.21.15.97:0/2780787089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100210 msgr2=0x7f3adc107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2780787089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100210 0x7f3adc107ad0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400b0a0 tx=0x7f3ad400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 -- 172.21.15.97:0/2780787089 shutdown_connections 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2780787089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100210 0x7f3adc107ad0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2780787089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc100b40 0x7f3adc100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2780787089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc0ff870 0x7f3adc0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 -- 172.21.15.97:0/2780787089 >> 172.21.15.97:0/2780787089 conn(0x7f3adc0fa2c0 msgr2=0x7f3adc0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 -- 172.21.15.97:0/2780787089 shutdown_connections 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.864+0000 7f3ae4297700 1 -- 172.21.15.97:0/2780787089 wait complete. 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 Processor -- start 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 -- start start 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 0x7f3adc196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc100210 0x7f3adc196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc19e270 con 0x7f3adc100b40 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc19e3b0 con 0x7f3adc100210 2023-10-08T09:55:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae4297700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc19e540 con 0x7f3adc0ff870 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56650/0 (socket says 172.21.15.97:56650) 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 -- 172.21.15.97:0/2303858202 learned_addr learned my addr 172.21.15.97:0/2303858202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae1832700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc100210 0x7f3adc196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2033700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 0x7f3adc196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 -- 172.21.15.97:0/2303858202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 msgr2=0x7f3adc196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 0x7f3adc196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 -- 172.21.15.97:0/2303858202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc100210 msgr2=0x7f3adc196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc100210 0x7f3adc196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 -- 172.21.15.97:0/2303858202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad4009d00 con 0x7f3adc100b40 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2033700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 0x7f3adc196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.865+0000 7f3ae2834700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4004d00 tx=0x7f3ad4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.866+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400e040 con 0x7f3adc100b40 2023-10-08T09:55:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.866+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ad400cec0 con 0x7f3adc100b40 2023-10-08T09:55:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.866+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc19e7c0 con 0x7f3adc100b40 2023-10-08T09:55:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.866+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401ba80 con 0x7f3adc100b40 2023-10-08T09:55:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.866+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc19ec40 con 0x7f3adc100b40 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.867+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc106a40 con 0x7f3adc100b40 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.868+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3ad401bbe0 con 0x7f3adc100b40 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.868+0000 7f3acf7fe700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 0x7f3ac80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.868+0000 7f3ae2033700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 0x7f3ac80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.868+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ad40ceea0 con 0x7f3adc100b40 2023-10-08T09:55:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.869+0000 7f3ae2033700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 0x7f3ac80a39c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005e10 tx=0x7f3ad0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:39.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:39.872+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3ad4098280 con 0x7f3adc100b40 2023-10-08T09:55:40.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.106+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3adc066bc0 con 0x7f3adc100b40 2023-10-08T09:55:40.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.107+0000 7f3acf7fe700 1 -- 172.21.15.97:0/2303858202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f3ad409b8a0 con 0x7f3adc100b40 2023-10-08T09:55:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:40.108 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:40.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 msgr2=0x7f3ac80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 0x7f3ac80a39c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005e10 tx=0x7f3ad0005d20 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 msgr2=0x7f3adc19dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4004d00 tx=0x7f3ad4004d30 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 shutdown_connections 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f3ac80a1500 0x7f3ac80a39c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3adc100b40 0x7f3adc19dca0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3adc0ff870 0x7f3adc196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 --2- 172.21.15.97:0/2303858202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3adc100210 0x7f3adc196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 >> 172.21.15.97:0/2303858202 conn(0x7f3adc0fa2c0 msgr2=0x7f3adc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 shutdown_connections 2023-10-08T09:55:40.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.109+0000 7f3ae4297700 1 -- 172.21.15.97:0/2303858202 wait complete. 2023-10-08T09:55:40.111 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:40.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 -- 172.21.15.97:0/1284200386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c00ff5b0 msgr2=0x7f59c00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/1284200386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c00ff5b0 0x7f59c00ff9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f59b40099c0 tx=0x7f59b4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 -- 172.21.15.97:0/1284200386 shutdown_connections 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/1284200386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c0101140 0x7f59c01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/1284200386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c00ff5b0 0x7f59c00ff9d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/1284200386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 0x7f59c0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 -- 172.21.15.97:0/1284200386 >> 172.21.15.97:0/1284200386 conn(0x7f59c00fab40 msgr2=0x7f59c00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 -- 172.21.15.97:0/1284200386 shutdown_connections 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.243+0000 7f59c4fd8700 1 -- 172.21.15.97:0/1284200386 wait complete. 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 Processor -- start 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 -- start start 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c00ff5b0 0x7f59c00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 0x7f59c019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c019e080 con 0x7f59c0101140 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c019e1f0 con 0x7f59c01007a0 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59c4fd8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c019e3b0 con 0x7f59c00ff5b0 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56664/0 (socket says 172.21.15.97:56664) 2023-10-08T09:55:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 -- 172.21.15.97:0/4064937672 learned_addr learned my addr 172.21.15.97:0/4064937672 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59be59c700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c00ff5b0 0x7f59c00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bdd9b700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 0x7f59c019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 -- 172.21.15.97:0/4064937672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c00ff5b0 msgr2=0x7f59c00695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c00ff5b0 0x7f59c00695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 -- 172.21.15.97:0/4064937672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 msgr2=0x7f59c019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 0x7f59c019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 -- 172.21.15.97:0/4064937672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40095f0 con 0x7f59c0101140 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.244+0000 7f59bed9d700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59b8007ee0 tx=0x7f59b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.245+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8017070 con 0x7f59c0101140 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.245+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b800f040 con 0x7f59c0101140 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.245+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c019e690 con 0x7f59c0101140 2023-10-08T09:55:40.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.245+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8005740 con 0x7f59c0101140 2023-10-08T09:55:40.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.245+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c019ebf0 con 0x7f59c0101140 2023-10-08T09:55:40.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.246+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c01069e0 con 0x7f59c0101140 2023-10-08T09:55:40.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.247+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f59b80058a0 con 0x7f59c0101140 2023-10-08T09:55:40.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.247+0000 7f59af7fe700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 0x7f59a80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.247+0000 7f59be59c700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 0x7f59a80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.247+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f59b80cc810 con 0x7f59c0101140 2023-10-08T09:55:40.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.248+0000 7f59be59c700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 0x7f59a80a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b580 tx=0x7f59b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.252+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59b8095b60 con 0x7f59c0101140 2023-10-08T09:55:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.444+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c0066bc0 con 0x7f59c0101140 2023-10-08T09:55:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.446+0000 7f59af7fe700 1 -- 172.21.15.97:0/4064937672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7f59b8099180 con 0x7f59c0101140 2023-10-08T09:55:40.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:40.447 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 msgr2=0x7f59a80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 0x7f59a80a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b580 tx=0x7f59b4005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 msgr2=0x7f59c019d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59b8007ee0 tx=0x7f59b800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 shutdown_connections 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f59a80a1500 0x7f59a80a39c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59c0101140 0x7f59c019d8e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59c00ff5b0 0x7f59c00695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 --2- 172.21.15.97:0/4064937672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01007a0 0x7f59c019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 >> 172.21.15.97:0/4064937672 conn(0x7f59c00fab40 msgr2=0x7f59c0103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 shutdown_connections 2023-10-08T09:55:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.448+0000 7f59c4fd8700 1 -- 172.21.15.97:0/4064937672 wait complete. 2023-10-08T09:55:40.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:40 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1460971427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:40 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3539601461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:40 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2303858202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:40.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T09:53:56.599415+0000', 'modified': '2023-10-08T09:54:24.185261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24562, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/2717102390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 2717102390}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 2717102390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/3587378302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 3587378302}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 3587378302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24536': {'gid': 24536, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/2015632800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2015632800}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2015632800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.186:6837/4190406762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 4190406762}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 4190406762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.186:6839/2120449823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2120449823}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2120449823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6839/1665352653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 1665352653}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 1665352653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T09:55:40.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-08T09:55:40.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T09:55:40.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:40 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1460971427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:40 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3539601461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:40 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2303858202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 -- 172.21.15.97:0/94967257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 msgr2=0x7fa4140ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 --2- 172.21.15.97:0/94967257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa4140ffad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa4040099c0 tx=0x7fa404009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 -- 172.21.15.97:0/94967257 shutdown_connections 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 --2- 172.21.15.97:0/94967257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa414101240 0x7fa414107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 --2- 172.21.15.97:0/94967257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa4140ffad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 --2- 172.21.15.97:0/94967257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4141008a0 0x7fa414100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 -- 172.21.15.97:0/94967257 >> 172.21.15.97:0/94967257 conn(0x7fa4140fac40 msgr2=0x7fa4140fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 -- 172.21.15.97:0/94967257 shutdown_connections 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.583+0000 7fa41a220700 1 -- 172.21.15.97:0/94967257 wait complete. 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 Processor -- start 2023-10-08T09:55:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 -- start start 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4141008a0 0x7fa414195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa414101240 0x7fa41419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41419d1a0 con 0x7fa4140ff6b0 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41419d2e0 con 0x7fa4141008a0 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa41a220700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41419d470 con 0x7fa414101240 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa4137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa4137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56674/0 (socket says 172.21.15.97:56674) 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa4137fe700 1 -- 172.21.15.97:0/2196909328 learned_addr learned my addr 172.21.15.97:0/2196909328 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.584+0000 7fa413fff700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa414101240 0x7fa41419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4141008a0 0x7fa414195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 -- 172.21.15.97:0/2196909328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa414101240 msgr2=0x7fa41419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa414101240 0x7fa41419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 -- 172.21.15.97:0/2196909328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4141008a0 msgr2=0x7fa414195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4141008a0 0x7fa414195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 -- 172.21.15.97:0/2196909328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040095f0 con 0x7fa4140ff6b0 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa404006010 tx=0x7fa404005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa410ff9700 1 -- 172.21.15.97:0/2196909328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40401d070 con 0x7fa4140ff6b0 2023-10-08T09:55:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa410ff9700 1 -- 172.21.15.97:0/2196909328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa404004010 con 0x7fa4140ff6b0 2023-10-08T09:55:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa410ff9700 1 -- 172.21.15.97:0/2196909328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404022cd0 con 0x7fa4140ff6b0 2023-10-08T09:55:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41419d6f0 con 0x7fa4140ff6b0 2023-10-08T09:55:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.585+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41419db50 con 0x7fa4140ff6b0 2023-10-08T09:55:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.587+0000 7fa410ff9700 1 -- 172.21.15.97:0/2196909328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa404022470 con 0x7fa4140ff6b0 2023-10-08T09:55:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.588+0000 7fa410ff9700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 0x7fa3fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.588+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 0x7fa3fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.588+0000 7fa410ff9700 1 -- 172.21.15.97:0/2196909328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4040ce500 con 0x7fa4140ff6b0 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.588+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 0x7fa3fc0a3940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa408005950 tx=0x7fa4080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 Processor -- start 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 -- start start 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb4495430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb44967b0 0x55fbb4496c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43fd410 con 0x55fbb4495970 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43eebd0 con 0x55fbb44967b0 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa426409b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43ec4f0 con 0x55fbb4495010 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56684/0 (socket says 172.21.15.97:56684) 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 -- 172.21.15.97:0/2432157390 learned_addr learned my addr 172.21.15.97:0/2432157390 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb4495430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa413fff700 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb44967b0 0x55fbb4496c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 -- 172.21.15.97:0/2432157390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 msgr2=0x55fbb4495430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb4495430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 -- 172.21.15.97:0/2432157390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb44967b0 msgr2=0x55fbb4496c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb44967b0 0x55fbb4496c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 -- 172.21.15.97:0/2432157390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb4212d90 con 0x55fbb4495970 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa40800fd70 tx=0x7fa408012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4e7ea18d7fff2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.589+0000 7fa3f9ffb700 1 -- 172.21.15.97:0/2432157390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408011c60 con 0x55fbb4495970 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa3f9ffb700 1 -- 172.21.15.97:0/2432157390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa408011dc0 con 0x55fbb4495970 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa3f9ffb700 1 -- 172.21.15.97:0/2432157390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40801d770 con 0x55fbb4495970 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 -- 172.21.15.97:0/2432157390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 msgr2=0x55fbb4496220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa40800fd70 tx=0x7fa408012000 comp rx=0 tx=0).stop 2023-10-08T09:55:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 -- 172.21.15.97:0/2432157390 shutdown_connections 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb44967b0 0x55fbb4496c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb4495430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 --2- 172.21.15.97:0/2432157390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495970 0x55fbb4496220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 -- 172.21.15.97:0/2432157390 >> 172.21.15.97:0/2432157390 conn(0x55fbb4409210 msgr2=0x55fbb448fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 -- 172.21.15.97:0/2432157390 shutdown_connections 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.590+0000 7fa426409b80 1 -- 172.21.15.97:0/2432157390 wait complete. 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 -- start start 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 Processor -- start 2023-10-08T09:55:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb449f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 0x55fbb44a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43e7de0 con 0x55fbb4495010 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43846c0 con 0x55fbb4495970 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb4384360 con 0x55fbb44967b0 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa426409b80 10 client.0 _ll_register_callbacks cb 0x55fbb41e17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56688/0 (socket says 172.21.15.97:56688) 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 -- 172.21.15.97:0/1693871636 learned_addr learned my addr 172.21.15.97:0/1693871636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa412ffd700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb449f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa413fff700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 0x55fbb44a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 -- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 msgr2=0x55fbb44a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 0x55fbb44a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 -- 172.21.15.97:0/1693871636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 msgr2=0x55fbb449f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.591+0000 7fa4137fe700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb449f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa4137fe700 1 -- 172.21.15.97:0/1693871636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40800fa20 con 0x55fbb4495010 2023-10-08T09:55:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa413fff700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 0x55fbb44a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa4137fe700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cbc90 tx=0x7fa4040227a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639b2eb933c5dae4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40409bc40 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa40409b540 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbb43afe60 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40409f690 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 10 client.15495 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fbb41fbc10 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbb43afe60 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa404084b70 con 0x55fbb4495010 2023-10-08T09:55:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 10 client.15495 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 20 client.15495 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb44aa520 0x55fbb44a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 4 client.15495 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.592+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbb43e0dc0 con 0x55fbb44aa520 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.593+0000 7fa412ffd700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb44aa520 0x55fbb44a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.593+0000 7fa412ffd700 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb44aa520 0x55fbb44a92d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd37619508f549f1 server_cookie=86ef915a98a3a99d in_seq=0 out_seq=0 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.593+0000 7fa3f2ffd700 10 client.15495 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/1693871636 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fbb43e0dc0 con 0x55fbb44aa520 2023-10-08T09:55:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa3f2ffd700 10 client.15495 handle_command_reply: tid=0 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 client.15495 shutdown 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 msgr2=0x55fbb449ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cbc90 tx=0x7fa4040227a0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 shutdown_connections 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb44aa520 0x55fbb44a92d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb44967b0 0x55fbb44a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa3f2ffd700 0 client.15495 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb4495010 0x55fbb449ed80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 --2- 172.21.15.97:0/1693871636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb449f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 >> 172.21.15.97:0/1693871636 conn(0x55fbb4409210 msgr2=0x55fbb448f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 shutdown_connections 2023-10-08T09:55:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 1 -- 172.21.15.97:0/1693871636 wait complete. 2023-10-08T09:55:40.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.594+0000 7fa426409b80 20 client.15495 trim_cache size 0 max 16384 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 Processor -- start 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 -- start start 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb42688d0 con 0x55fbb44967b0 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43afe60 con 0x55fbb4495970 2023-10-08T09:55:40.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.611+0000 7fa426409b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb43e9070 con 0x55fbb4495010 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56700/0 (socket says 172.21.15.97:56700) 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 -- 172.21.15.97:0/1499659857 learned_addr learned my addr 172.21.15.97:0/1499659857 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa4137fe700 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa412ffd700 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 -- 172.21.15.97:0/1499659857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 msgr2=0x55fbb44a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 -- 172.21.15.97:0/1499659857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 msgr2=0x55fbb44996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 -- 172.21.15.97:0/1499659857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb4239200 con 0x55fbb44967b0 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa413fff700 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00a480 tx=0x7fa40c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d9ea239bce86fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa3fa7fc700 1 -- 172.21.15.97:0/1499659857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ade0 con 0x55fbb44967b0 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa3fa7fc700 1 -- 172.21.15.97:0/1499659857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40c011050 con 0x55fbb44967b0 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.612+0000 7fa3fa7fc700 1 -- 172.21.15.97:0/1499659857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0167b0 con 0x55fbb44967b0 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- 172.21.15.97:0/1499659857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 msgr2=0x55fbb4499c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00a480 tx=0x7fa40c009650 comp rx=0 tx=0).stop 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- 172.21.15.97:0/1499659857 shutdown_connections 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb4499c10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- 172.21.15.97:0/1499659857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- 172.21.15.97:0/1499659857 >> 172.21.15.97:0/1499659857 conn(0x55fbb4409210 msgr2=0x55fbb449e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- 172.21.15.97:0/1499659857 shutdown_connections 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- 172.21.15.97:0/1499659857 wait complete. 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- start start 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 Processor -- start 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb44a4f80 con 0x55fbb44967b0 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb44a50f0 con 0x55fbb4495970 2023-10-08T09:55:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb44a52b0 con 0x55fbb4495010 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa426409b80 10 client.0 _ll_register_callbacks cb 0x7fa40409b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa413fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.613+0000 7fa413fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56706/0 (socket says 172.21.15.97:56706) 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 -- 172.21.15.97:0/2356568298 learned_addr learned my addr 172.21.15.97:0/2356568298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 -- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 msgr2=0x55fbb44a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 -- 172.21.15.97:0/2356568298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 msgr2=0x55fbb44a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 -- 172.21.15.97:0/2356568298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c009d20 con 0x55fbb44967b0 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa412ffd700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa413fff700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa40c004b10 tx=0x7fa40c017840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a6ea77203d0216 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa3f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02b670 con 0x55fbb44967b0 2023-10-08T09:55:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa40c006e40 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa408011dc0 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40c02b7d0 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa426409b80 10 client.15501 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fbb4211260 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.614+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa408011dc0 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa40c016a50 con 0x55fbb44967b0 2023-10-08T09:55:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 10 client.15501 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 20 client.15501 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb4507ff0 0x55fbb4504450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 4 client.15501 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fbb43e0ae0 con 0x55fbb4507ff0 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb4507ff0 0x55fbb4504450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa3f2ffd700 10 client.15501 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa4137fe700 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb4507ff0 0x55fbb4504450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6f4f450fbd77f65 server_cookie=26e4ae6cc35c9c0e in_seq=0 out_seq=0 2023-10-08T09:55:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa3f2ffd700 1 -- 172.21.15.97:0/2356568298 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fbb43e0ae0 con 0x55fbb4507ff0 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa3f2ffd700 10 client.15501 handle_command_reply: tid=0 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.615+0000 7fa426409b80 1 client.15501 shutdown 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 msgr2=0x55fbb44a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa40c004b10 tx=0x7fa40c017840 comp rx=0 tx=0).stop 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 shutdown_connections 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55fbb4507ff0 0x55fbb4504450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fbb44967b0 0x55fbb44a4a40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fbb4495010 0x55fbb44a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa3f2ffd700 0 client.15501 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 --2- 172.21.15.97:0/2356568298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fbb4495970 0x55fbb44a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 >> 172.21.15.97:0/2356568298 conn(0x55fbb4409210 msgr2=0x55fbb44a92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 shutdown_connections 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 1 -- 172.21.15.97:0/2356568298 wait complete. 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa426409b80 20 client.15501 trim_cache size 0 max 16384 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:40.627 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24541, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 77.435162607999999, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 120.64318992600001 2023-10-08T09:55:40.628 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 msgr2=0x7fa3fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 0x7fa3fc0a3940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa408005950 tx=0x7fa4080058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 msgr2=0x7fa414195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa404006010 tx=0x7fa404005740 comp rx=0 tx=0).stop 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 shutdown_connections 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fa3fc0a1480 0x7fa3fc0a3940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa414101240 0x7fa41419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4140ff6b0 0x7fa414195670 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 --2- 172.21.15.97:0/2196909328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4141008a0 0x7fa414195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 >> 172.21.15.97:0/2196909328 conn(0x7fa4140fac40 msgr2=0x7fa414103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 shutdown_connections 2023-10-08T09:55:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.616+0000 7fa41a220700 1 -- 172.21.15.97:0/2196909328 wait complete. 2023-10-08T09:55:40.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T09:55:40.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:40 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1460971427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:40 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3539601461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:40.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:40 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2303858202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 -- 172.21.15.97:0/1752003206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed84101230 msgr2=0x7fed84107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 --2- 172.21.15.97:0/1752003206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed84101230 0x7fed84107ae0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00b0d0 tx=0x7fed7c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 -- 172.21.15.97:0/1752003206 shutdown_connections 2023-10-08T09:55:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 --2- 172.21.15.97:0/1752003206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed84101230 0x7fed84107ae0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 --2- 172.21.15.97:0/1752003206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed840ff6a0 0x7fed840ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 --2- 172.21.15.97:0/1752003206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84100890 0x7fed84100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 -- 172.21.15.97:0/1752003206 >> 172.21.15.97:0/1752003206 conn(0x7fed840fac10 msgr2=0x7fed840fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 -- 172.21.15.97:0/1752003206 shutdown_connections 2023-10-08T09:55:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.752+0000 7fed8907f700 1 -- 172.21.15.97:0/1752003206 wait complete. 2023-10-08T09:55:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 Processor -- start 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 -- start start 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 0x7fed84195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84101230 0x7fed8419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8419d220 con 0x7fed840ff6a0 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8419d360 con 0x7fed84101230 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8907f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8419d4f0 con 0x7fed84100890 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56716/0 (socket says 172.21.15.97:56716) 2023-10-08T09:55:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 -- 172.21.15.97:0/286671863 learned_addr learned my addr 172.21.15.97:0/286671863 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8359e700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84101230 0x7fed8419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8259c700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 0x7fed84195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 -- 172.21.15.97:0/286671863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 msgr2=0x7fed84195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 0x7fed84195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 -- 172.21.15.97:0/286671863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84101230 msgr2=0x7fed8419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84101230 0x7fed8419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed82d9d700 1 -- 172.21.15.97:0/286671863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009d00 con 0x7fed840ff6a0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.753+0000 7fed8259c700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 0x7fed84195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed82d9d700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fed7800ea80 tx=0x7fed7800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed6bfff700 1 -- 172.21.15.97:0/286671863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78013070 con 0x7fed840ff6a0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed84067890 con 0x7fed840ff6a0 2023-10-08T09:55:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed6bfff700 1 -- 172.21.15.97:0/286671863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7800be80 con 0x7fed840ff6a0 2023-10-08T09:55:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed6bfff700 1 -- 172.21.15.97:0/286671863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78003ae0 con 0x7fed840ff6a0 2023-10-08T09:55:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.754+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84067df0 con 0x7fed840ff6a0 2023-10-08T09:55:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.756+0000 7fed6bfff700 1 -- 172.21.15.97:0/286671863 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fed780041b0 con 0x7fed840ff6a0 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.756+0000 7fed6bfff700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 0x7fed6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.756+0000 7fed8259c700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 0x7fed6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.756+0000 7fed6bfff700 1 -- 172.21.15.97:0/286671863 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed780ce820 con 0x7fed840ff6a0 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.757+0000 7fed8259c700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 0x7fed6c0a3940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed74009cb0 tx=0x7fed74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.757+0000 7fed95268b80 1 Processor -- start 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.757+0000 7fed95268b80 1 -- start start 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fc9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 0x564679fca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679f31410 con 0x564679fca7b0 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679f22bd0 con 0x564679fc9970 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed95268b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679f204f0 con 0x564679fc9010 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56728/0 (socket says 172.21.15.97:56728) 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 -- 172.21.15.97:0/1510225954 learned_addr learned my addr 172.21.15.97:0/1510225954 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8259c700 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 0x564679fca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 -- 172.21.15.97:0/1510225954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 msgr2=0x564679fc9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fc9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 -- 172.21.15.97:0/1510225954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 msgr2=0x564679fca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 0x564679fca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 -- 172.21.15.97:0/1510225954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564679d46d90 con 0x564679fca7b0 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8259c700 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 0x564679fca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed8359e700 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed7c015000 tx=0x7fed7c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e649f1c06996b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed697fa700 1 -- 172.21.15.97:0/1510225954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00e040 con 0x564679fca7b0 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed697fa700 1 -- 172.21.15.97:0/1510225954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7c00baa0 con 0x564679fca7b0 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.758+0000 7fed697fa700 1 -- 172.21.15.97:0/1510225954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c005070 con 0x564679fca7b0 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- 172.21.15.97:0/1510225954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 msgr2=0x564679fcac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed7c015000 tx=0x7fed7c009c20 comp rx=0 tx=0).stop 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- 172.21.15.97:0/1510225954 shutdown_connections 2023-10-08T09:55:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fcac40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fc9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 --2- 172.21.15.97:0/1510225954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9970 0x564679fca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- 172.21.15.97:0/1510225954 >> 172.21.15.97:0/1510225954 conn(0x564679f3d210 msgr2=0x564679fc3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- 172.21.15.97:0/1510225954 shutdown_connections 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- 172.21.15.97:0/1510225954 wait complete. 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.759+0000 7fed95268b80 1 -- start start 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 Processor -- start 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 0x564679fd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679f1bde0 con 0x564679fca7b0 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679eb86c0 con 0x564679fc9010 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679eb8360 con 0x564679fc9970 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed95268b80 10 client.0 _ll_register_callbacks cb 0x564679d157b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56734/0 (socket says 172.21.15.97:56734) 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 -- 172.21.15.97:0/4254158504 learned_addr learned my addr 172.21.15.97:0/4254158504 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed82d9d700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8259c700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 0x564679fd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 -- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 msgr2=0x564679fd32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 0x564679fd32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 -- 172.21.15.97:0/4254158504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 msgr2=0x564679fd2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8359e700 1 -- 172.21.15.97:0/4254158504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84108430 con 0x564679fca7b0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed8259c700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 0x564679fd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.760+0000 7fed82d9d700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed8359e700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007b60 tx=0x7fed7c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5daa3d0130a2590 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00e040 con 0x564679fca7b0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed7c0052d0 con 0x564679fca7b0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564679ee3e60 con 0x564679fca7b0 2023-10-08T09:55:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed7c005490 con 0x564679fca7b0 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 10 client.15513 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564679d2fc10 con 0x564679fca7b0 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564679ee3e60 con 0x564679fca7b0 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed7c00b100 con 0x564679fca7b0 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 10 client.15513 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 20 client.15513 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x564679fde4f0 0x564679fdd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 4 client.15513 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.761+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564679f14dc0 con 0x564679fde4f0 2023-10-08T09:55:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.762+0000 7fed82d9d700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x564679fde4f0 0x564679fdd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.762+0000 7fed66ff5700 10 client.15513 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.762+0000 7fed82d9d700 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x564679fde4f0 0x564679fdd250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca62f6f524fe21e8 server_cookie=ed11c82920949c29 in_seq=0 out_seq=0 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed66ff5700 1 -- 172.21.15.97:0/4254158504 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564679f14dc0 con 0x564679fde4f0 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed66ff5700 10 client.15513 handle_command_reply: tid=0 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 client.15513 shutdown 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 msgr2=0x564679fda310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007b60 tx=0x7fed7c007780 comp rx=0 tx=0).stop 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 shutdown_connections 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fca7b0 0x564679fda310 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x564679fde4f0 0x564679fdd250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed66ff5700 0 client.15513 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 --2- 172.21.15.97:0/4254158504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9970 0x564679fd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 >> 172.21.15.97:0/4254158504 conn(0x564679f3d210 msgr2=0x564679fc3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 shutdown_connections 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 1 -- 172.21.15.97:0/4254158504 wait complete. 2023-10-08T09:55:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.763+0000 7fed95268b80 20 client.15513 trim_cache size 0 max 16384 2023-10-08T09:55:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.780+0000 7fed95268b80 1 Processor -- start 2023-10-08T09:55:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.780+0000 7fed95268b80 1 -- start start 2023-10-08T09:55:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fca7b0 0x564679fc3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679d9c8d0 con 0x564679fc9970 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679f1bde0 con 0x564679fc9010 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed95268b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679fc4ab0 con 0x564679fca7b0 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56740/0 (socket says 172.21.15.97:56740) 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 -- 172.21.15.97:0/4282029564 learned_addr learned my addr 172.21.15.97:0/4282029564 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8359e700 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fca7b0 0x564679fc3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 -- 172.21.15.97:0/4282029564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fca7b0 msgr2=0x564679fc3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fca7b0 0x564679fc3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed82d9d700 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 -- 172.21.15.97:0/4282029564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 msgr2=0x564679fd22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 -- 172.21.15.97:0/4282029564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564679d6d200 con 0x564679fc9970 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed8259c700 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fed74009b70 tx=0x7fed74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d144679e7ed7ece server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed69ffb700 1 -- 172.21.15.97:0/4282029564 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003a00 con 0x564679fc9970 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed69ffb700 1 -- 172.21.15.97:0/4282029564 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed74003b60 con 0x564679fc9970 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.781+0000 7fed69ffb700 1 -- 172.21.15.97:0/4282029564 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74020660 con 0x564679fc9970 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- 172.21.15.97:0/4282029564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 msgr2=0x564679fda230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fed74009b70 tx=0x7fed74014000 comp rx=0 tx=0).stop 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- 172.21.15.97:0/4282029564 shutdown_connections 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fca7b0 0x564679fc3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fc9010 0x564679fd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- 172.21.15.97:0/4282029564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fda230 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- 172.21.15.97:0/4282029564 >> 172.21.15.97:0/4282029564 conn(0x564679f3d210 msgr2=0x564679fbfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- 172.21.15.97:0/4282029564 shutdown_connections 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- 172.21.15.97:0/4282029564 wait complete. 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- start start 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 Processor -- start 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fda0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fca7b0 0x564679fd7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564679fd79d0 con 0x564679fc9970 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467a031ee0 con 0x564679fca7b0 2023-10-08T09:55:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.782+0000 7fed95268b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56467a032050 con 0x564679fc9010 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed95268b80 10 client.0 _ll_register_callbacks cb 0x7fed7c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56746/0 (socket says 172.21.15.97:56746) 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 -- 172.21.15.97:0/92812330 learned_addr learned my addr 172.21.15.97:0/92812330 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed82d9d700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fda0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 -- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 msgr2=0x564679fda0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fda0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 -- 172.21.15.97:0/92812330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fca7b0 msgr2=0x564679fd7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fca7b0 0x564679fd7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 -- 172.21.15.97:0/92812330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed74011d40 con 0x564679fc9970 2023-10-08T09:55:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed8259c700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed74009fd0 tx=0x7fed74003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09ca77e1700965c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7402e070 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed7c00baa0 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed74018070 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed74012660 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed95268b80 10 client.15519 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564679d45260 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.783+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7c00baa0 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed74004af0 con 0x564679fc9970 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 10 client.15519 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:55:40.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 20 client.15519 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x56467a0373c0 0x56467a033830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 4 client.15519 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564679f14ae0 con 0x56467a0373c0 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed82d9d700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x56467a0373c0 0x56467a033830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed82d9d700 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x56467a0373c0 0x56467a033830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1ce53e1fa93a8e7 server_cookie=62f0f319e60dff9a in_seq=0 out_seq=0 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.784+0000 7fed66ff5700 10 client.15519 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed66ff5700 1 -- 172.21.15.97:0/92812330 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564679f14ae0 con 0x56467a0373c0 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed66ff5700 10 client.15519 handle_command_reply: tid=0 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 client.15519 shutdown 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 msgr2=0x564679fd6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fed74009fd0 tx=0x7fed74003a80 comp rx=0 tx=0).stop 2023-10-08T09:55:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 shutdown_connections 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x56467a0373c0 0x56467a033830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564679fca7b0 0x564679fd7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed66ff5700 0 client.15519 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564679fc9010 0x564679fda0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 --2- 172.21.15.97:0/92812330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564679fc9970 0x564679fd6d50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 >> 172.21.15.97:0/92812330 conn(0x564679f3d210 msgr2=0x564679fd2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 shutdown_connections 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 1 -- 172.21.15.97:0/92812330 wait complete. 2023-10-08T09:55:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed95268b80 20 client.15519 trim_cache size 0 max 16384 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24562, 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 78.614043197000001, 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 116.611823617 2023-10-08T09:55:40.795 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:40.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 msgr2=0x7fed6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 0x7fed6c0a3940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed74009cb0 tx=0x7fed74009260 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 msgr2=0x7fed841956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fed7800ea80 tx=0x7fed7800ed90 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 shutdown_connections 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed6c0a1480 0x7fed6c0a3940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed84101230 0x7fed8419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed840ff6a0 0x7fed841956f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 --2- 172.21.15.97:0/286671863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed84100890 0x7fed84195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.785+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 >> 172.21.15.97:0/286671863 conn(0x7fed840fac10 msgr2=0x7fed84103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.786+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 shutdown_connections 2023-10-08T09:55:40.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.786+0000 7fed8907f700 1 -- 172.21.15.97:0/286671863 wait complete. 2023-10-08T09:55:40.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:55:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.919+0000 7fe5744f9700 1 -- 172.21.15.97:0/2972934632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c1008c0 msgr2=0x7fe56c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.919+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2972934632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c1008c0 0x7fe56c100d20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0099c0 tx=0x7fe55c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- 172.21.15.97:0/2972934632 shutdown_connections 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2972934632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 0x7fe56c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2972934632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2972934632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c1008c0 0x7fe56c100d20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- 172.21.15.97:0/2972934632 >> 172.21.15.97:0/2972934632 conn(0x7fe56c0fac40 msgr2=0x7fe56c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- 172.21.15.97:0/2972934632 shutdown_connections 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- 172.21.15.97:0/2972934632 wait complete. 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 Processor -- start 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- start start 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 0x7fe56c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 0x7fe56c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c19e240 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c19e3b0 con 0x7fe56c1008c0 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe5744f9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c19e570 con 0x7fe56c101260 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe572295700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.920+0000 7fe572295700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56758/0 (socket says 172.21.15.97:56758) 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 -- 172.21.15.97:0/2508761850 learned_addr learned my addr 172.21.15.97:0/2508761850 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572a96700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 0x7fe56c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe571a94700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 0x7fe56c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 -- 172.21.15.97:0/2508761850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 msgr2=0x7fe56c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 0x7fe56c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 -- 172.21.15.97:0/2508761850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 msgr2=0x7fe56c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 0x7fe56c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 -- 172.21.15.97:0/2508761850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0095f0 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe571a94700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 0x7fe56c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe572295700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe56800d980 tx=0x7fe56800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe5637fe700 1 -- 172.21.15.97:0/2508761850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568004020 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c19e7f0 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe5637fe700 1 -- 172.21.15.97:0/2508761850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe568009730 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe5637fe700 1 -- 172.21.15.97:0/2508761850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568003a30 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.921+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c19ed20 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.923+0000 7fe5637fe700 1 -- 172.21.15.97:0/2508761850 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe5680098a0 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.924+0000 7fe5637fe700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 0x7fe5580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.924+0000 7fe571a94700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 0x7fe5580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.924+0000 7fe5637fe700 1 -- 172.21.15.97:0/2508761850 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5680ccd40 con 0x7fe56c0ff6d0 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.924+0000 7fe571a94700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 0x7fe5580a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe55c000c00 tx=0x7fe55c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 Processor -- start 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 -- start start 2023-10-08T09:55:40.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826010 0x5620d9826430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d9827220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d978e410 con 0x5620d98277b0 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d977fbd0 con 0x5620d9826970 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d977d4f0 con 0x5620d9826010 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56770/0 (socket says 172.21.15.97:56770) 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 -- 172.21.15.97:0/945505353 learned_addr learned my addr 172.21.15.97:0/945505353 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572295700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826010 0x5620d9826430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe571a94700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d9827220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 -- 172.21.15.97:0/945505353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826010 msgr2=0x5620d9826430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826010 0x5620d9826430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 -- 172.21.15.97:0/945505353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 msgr2=0x5620d9827220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d9827220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe572a96700 1 -- 172.21.15.97:0/945505353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620d95a3d90 con 0x5620d98277b0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.925+0000 7fe571a94700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d9827220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe572a96700 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe5640098f0 tx=0x7fe564009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4400523052279e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe560ff9700 1 -- 172.21.15.97:0/945505353 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564019070 con 0x5620d98277b0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe560ff9700 1 -- 172.21.15.97:0/945505353 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56400f040 con 0x5620d98277b0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe560ff9700 1 -- 172.21.15.97:0/945505353 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564007b30 con 0x5620d98277b0 2023-10-08T09:55:40.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 -- 172.21.15.97:0/945505353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 msgr2=0x5620d9827c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe5640098f0 tx=0x7fe564009c00 comp rx=0 tx=0).stop 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 -- 172.21.15.97:0/945505353 shutdown_connections 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d9827c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826010 0x5620d9826430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/945505353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d9827220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.926+0000 7fe5806e2b80 1 -- 172.21.15.97:0/945505353 >> 172.21.15.97:0/945505353 conn(0x5620d979a210 msgr2=0x5620d9820db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- 172.21.15.97:0/945505353 shutdown_connections 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- 172.21.15.97:0/945505353 wait complete. 2023-10-08T09:55:40.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- start start 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 Processor -- start 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d982fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d9830290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d9778de0 con 0x5620d98277b0 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d97156c0 con 0x5620d9826010 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d9715360 con 0x5620d9826970 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe5806e2b80 10 client.0 _ll_register_callbacks cb 0x5620d95727b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.927+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56776/0 (socket says 172.21.15.97:56776) 2023-10-08T09:55:40.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 -- 172.21.15.97:0/2962417148 learned_addr learned my addr 172.21.15.97:0/2962417148 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe571a94700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d9830290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572295700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d982fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 -- 172.21.15.97:0/2962417148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 msgr2=0x5620d9830290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d9830290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 -- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 msgr2=0x5620d982fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d982fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 -- 172.21.15.97:0/2962417148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe564003c50 con 0x5620d98277b0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572295700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d982fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe572a96700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5640047b0 tx=0x7fe5640047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af8f0222ed97bfb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564019040 con 0x5620d98277b0 2023-10-08T09:55:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe564007cc0 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620d9740e60 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe564007e80 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe5806e2b80 10 client.15531 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5620d958cc10 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.928+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620d9740e60 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe56401f750 con 0x5620d98277b0 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 20 client.15531 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d983b860 0x5620d983a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 4 client.15531 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620d9771dc0 con 0x5620d983b860 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe572295700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d983b860 0x5620d983a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe555ffb700 10 client.15531 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.929+0000 7fe572295700 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d983b860 0x5620d983a320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b298adabc442da server_cookie=6a30b1e29d1c0824 in_seq=0 out_seq=0 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.930+0000 7fe555ffb700 1 -- 172.21.15.97:0/2962417148 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620d9771dc0 con 0x5620d983b860 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.930+0000 7fe555ffb700 10 client.15531 handle_command_reply: tid=0 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.930+0000 7fe5806e2b80 1 client.15531 shutdown 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.930+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 msgr2=0x5620d98372d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.930+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5640047b0 tx=0x7fe5640047e0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 shutdown_connections 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d98372d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d982fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d983b860 0x5620d983a320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/2962417148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d9830290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe555ffb700 0 client.15531 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 >> 172.21.15.97:0/2962417148 conn(0x5620d979a210 msgr2=0x5620d9820480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 shutdown_connections 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 1 -- 172.21.15.97:0/2962417148 wait complete. 2023-10-08T09:55:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.931+0000 7fe5806e2b80 20 client.15531 trim_cache size 0 max 16384 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 Processor -- start 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 -- start start 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d981cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d98371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d95f98d0 con 0x5620d98277b0 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d9740e60 con 0x5620d9826010 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d971f4d0 con 0x5620d9826970 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe572a96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56784/0 (socket says 172.21.15.97:56784) 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.948+0000 7fe572a96700 1 -- 172.21.15.97:0/3015272206 learned_addr learned my addr 172.21.15.97:0/3015272206 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe571a94700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d98371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572295700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d981cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 -- 172.21.15.97:0/3015272206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 msgr2=0x5620d98371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d98371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 -- 172.21.15.97:0/3015272206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 msgr2=0x5620d981cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d981cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 -- 172.21.15.97:0/3015272206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620d95ca200 con 0x5620d98277b0 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe571a94700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d98371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe572a96700 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe564003f40 tx=0x7fe564009140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d2d6d69ee440ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe5617fa700 1 -- 172.21.15.97:0/3015272206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564019070 con 0x5620d98277b0 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe5617fa700 1 -- 172.21.15.97:0/3015272206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56401f450 con 0x5620d98277b0 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe5617fa700 1 -- 172.21.15.97:0/3015272206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640159a0 con 0x5620d98277b0 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe5806e2b80 1 -- 172.21.15.97:0/3015272206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 msgr2=0x5620d982a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.949+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe564003f40 tx=0x7fe564009140 comp rx=0 tx=0).stop 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- 172.21.15.97:0/3015272206 shutdown_connections 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d98277b0 0x5620d982a6d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826010 0x5620d981cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/3015272206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d9826970 0x5620d98371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- 172.21.15.97:0/3015272206 >> 172.21.15.97:0/3015272206 conn(0x5620d979a210 msgr2=0x5620d98441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- 172.21.15.97:0/3015272206 shutdown_connections 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- 172.21.15.97:0/3015272206 wait complete. 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- start start 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 Processor -- start 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d98323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d98277b0 0x5620d988f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d9837340 con 0x5620d9826010 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d98374b0 con 0x5620d9826970 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d988f790 con 0x5620d98277b0 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe5806e2b80 10 client.0 _ll_register_callbacks cb 0x5620d95727b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56786/0 (socket says 172.21.15.97:56786) 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 -- 172.21.15.97:0/829527348 learned_addr learned my addr 172.21.15.97:0/829527348 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe571a94700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d98323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572a96700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d98277b0 0x5620d988f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 -- 172.21.15.97:0/829527348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d98277b0 msgr2=0x5620d988f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d98277b0 0x5620d988f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 -- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 msgr2=0x5620d98323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.950+0000 7fe572295700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d98323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe572295700 1 -- 172.21.15.97:0/829527348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620d958cc10 con 0x5620d9826010 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe572295700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe568096760 tx=0x7fe568009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765fcaff3c932e06 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56809de40 con 0x5620d9826010 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5680a2080 con 0x5620d9826010 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe56400f040 con 0x5620d9826010 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe568009b80 con 0x5620d9826010 2023-10-08T09:55:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 10 client.15537 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5620d95a2260 con 0x5620d9826010 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56400f040 con 0x5620d9826010 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe5680992e0 con 0x5620d9826010 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 10 client.15537 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 20 client.15537 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d98949a0 0x5620d9890f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 4 client.15537 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.951+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5620d9771ae0 con 0x5620d98949a0 2023-10-08T09:55:40.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe571a94700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d98949a0 0x5620d9890f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe571a94700 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d98949a0 0x5620d9890f00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd807233a194c0ab server_cookie=55bc8d8a44f02814 in_seq=0 out_seq=0 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe555ffb700 10 client.15537 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe555ffb700 1 -- 172.21.15.97:0/829527348 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5620d9771ae0 con 0x5620d98949a0 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe555ffb700 10 client.15537 handle_command_reply: tid=0 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 client.15537 shutdown 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 msgr2=0x5620d9831e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe568096760 tx=0x7fe568009650 comp rx=0 tx=0).stop 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 shutdown_connections 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5620d98949a0 0x5620d9890f00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5620d98277b0 0x5620d988f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe555ffb700 0 client.15537 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620d9826010 0x5620d9831e70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 --2- 172.21.15.97:0/829527348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5620d9826970 0x5620d98323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 >> 172.21.15.97:0/829527348 conn(0x5620d979a210 msgr2=0x5620d981cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.952+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 shutdown_connections 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5806e2b80 1 -- 172.21.15.97:0/829527348 wait complete. 2023-10-08T09:55:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5806e2b80 20 client.15537 trim_cache size 0 max 16384 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "7ec07914-65bf-11ee-8db5-212e2dc638e7", 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24566, 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 103.752998875, 2023-10-08T09:55:40.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2023-10-08T09:55:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 87, 2023-10-08T09:55:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:55:40.964 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 109.899038722 2023-10-08T09:55:40.964 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 msgr2=0x7fe5580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 0x7fe5580a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe55c000c00 tx=0x7fe55c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 msgr2=0x7fe56c0694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe56800d980 tx=0x7fe56800dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 shutdown_connections 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5580a1480 0x7fe5580a3940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe56c101260 0x7fe56c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe56c0ff6d0 0x7fe56c0694d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 --2- 172.21.15.97:0/2508761850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe56c1008c0 0x7fe56c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 >> 172.21.15.97:0/2508761850 conn(0x7fe56c0fac40 msgr2=0x7fe56c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 shutdown_connections 2023-10-08T09:55:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:40.953+0000 7fe5744f9700 1 -- 172.21.15.97:0/2508761850 wait complete. 2023-10-08T09:55:40.966 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T09:55:40.982 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T09:55:40.982 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-08T09:55:40.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:40.983 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-10-08T09:55:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-08T09:55:41.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:41.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:55:41.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:41.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-10-08T09:55:41.178 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1'} 2023-10-08T09:55:41.178 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T09:55:41.178 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:41.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T09:55:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:41 smithi097 ceph-mon[99491]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:41.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:41 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4064937672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:41 smithi186 ceph-mon[104393]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:41 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4064937672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:41 smithi178 ceph-mon[103156]: pgmap v314: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:41 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4064937672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:41.738 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:55:43.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 -- 172.21.15.97:0/3142031548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 msgr2=0x7fdda4068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 --2- 172.21.15.97:0/3142031548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda4068f10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdd940099c0 tx=0x7fdd94009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 -- 172.21.15.97:0/3142031548 shutdown_connections 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 --2- 172.21.15.97:0/3142031548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4069450 0x7fdda410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 --2- 172.21.15.97:0/3142031548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4105670 0x7fdda4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 --2- 172.21.15.97:0/3142031548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda4068f10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 -- 172.21.15.97:0/3142031548 >> 172.21.15.97:0/3142031548 conn(0x7fdda40766c0 msgr2=0x7fdda4076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 -- 172.21.15.97:0/3142031548 shutdown_connections 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.026+0000 7fdda8af1700 1 -- 172.21.15.97:0/3142031548 wait complete. 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 Processor -- start 2023-10-08T09:55:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 -- start start 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4069450 0x7fdda419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4105670 0x7fdda41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4197030 con 0x7fdda4068a90 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda41971a0 con 0x7fdda4069450 2023-10-08T09:55:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda8af1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4197360 con 0x7fdda4105670 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56800/0 (socket says 172.21.15.97:56800) 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda259c700 1 -- 172.21.15.97:0/2784501874 learned_addr learned my addr 172.21.15.97:0/2784501874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda2d9d700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4105670 0x7fdda41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.027+0000 7fdda1d9b700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4069450 0x7fdda419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 -- 172.21.15.97:0/2784501874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4105670 msgr2=0x7fdda41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4105670 0x7fdda41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 -- 172.21.15.97:0/2784501874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4069450 msgr2=0x7fdda419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4069450 0x7fdda419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 -- 172.21.15.97:0/2784501874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940095f0 con 0x7fdda4068a90 2023-10-08T09:55:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda259c700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c9b0 tx=0x7fdd9800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98007b00 con 0x7fdda4068a90 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda4197640 con 0x7fdda4068a90 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd98007c60 con 0x7fdda4068a90 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98018660 con 0x7fdda4068a90 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.028+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda41a3830 con 0x7fdda4068a90 2023-10-08T09:55:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.029+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda4108d30 con 0x7fdda4068a90 2023-10-08T09:55:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.030+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdd9800f450 con 0x7fdda4068a90 2023-10-08T09:55:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.030+0000 7fdd937fe700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 0x7fdd8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.031+0000 7fdda1d9b700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 0x7fdd8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.031+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdd980ccf20 con 0x7fdda4068a90 2023-10-08T09:55:43.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.031+0000 7fdda1d9b700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 0x7fdd8c0a3940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdda4197b40 tx=0x7fdd94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:43.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.035+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd98096270 con 0x7fdda4068a90 2023-10-08T09:55:43.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.268+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdda4101ee0 con 0x7fdda4068a90 2023-10-08T09:55:43.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.269+0000 7fdd937fe700 1 -- 172.21.15.97:0/2784501874 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdd98099890 con 0x7fdda4068a90 2023-10-08T09:55:43.270 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T09:55:43.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 msgr2=0x7fdd8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 0x7fdd8c0a3940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdda4197b40 tx=0x7fdd94005fd0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 msgr2=0x7fdda419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c9b0 tx=0x7fdd9800ccc0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 shutdown_connections 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd8c0a1480 0x7fdd8c0a3940 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdda4105670 0x7fdda41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdda4068a90 0x7fdda419c840 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 --2- 172.21.15.97:0/2784501874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4069450 0x7fdda419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 >> 172.21.15.97:0/2784501874 conn(0x7fdda40766c0 msgr2=0x7fdda40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 shutdown_connections 2023-10-08T09:55:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:43.271+0000 7fdda8af1700 1 -- 172.21.15.97:0/2784501874 wait complete. 2023-10-08T09:55:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:43 smithi186 ceph-mon[104393]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:43.615 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:43 smithi097 ceph-mon[99491]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:43.697 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T09:55:43.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T09:55:43.698 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T09:55:43.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:43.698 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-10-08T09:55:43.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:43 smithi178 ceph-mon[103156]: pgmap v315: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-10-08T09:55:43.717 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41822sec preferred_lft 41822sec 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7efb:78c9:efd1:4682/64 scope link noprefixroute 2023-10-08T09:55:43.718 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:55:43.718 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T09:55:43.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T09:55:43.719 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:55:43.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:43.838 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T09:55:43.865 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-10-08T09:55:43.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:43.866 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-10-08T09:55:43.925 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-10-08T09:55:43.925 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T09:55:43.925 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T09:55:43.925 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T09:55:43.925 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T09:55:43.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T09:55:43.926 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:55:44.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.032 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-10-08T09:55:44.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.049 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-10-08T09:55:44.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.107 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:55:44.107 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:55:44.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T09:55:44.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T09:55:44.110 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:55:44.202 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T09:55:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.202 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:55:44.202 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:55:44.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T09:55:44.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T09:55:44.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T09:55:44.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T09:55:44.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T09:55:44.203 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:55:44.257 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:44 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2784501874' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:55:44.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.399 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-10-08T09:55:44.399 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-10-08T09:55:44.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-10-08T09:55:44.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T09:55:44.400 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-10-08T09:55:44.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:55:44.517 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:44.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:44 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2784501874' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:55:44.539 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T09:55:44.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:55:44.540 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:44.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-08T09:55:44.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-10-08T09:55:44.682 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-10-08T09:55:44.683 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-10-08T09:55:44.683 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2023-10-08T09:55:44.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:44 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2784501874' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.178:3300,172.21.15.186:3300" 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:3300,172.21.15.178:3300,172.21.15.186:3300:/ 2023-10-08T09:55:44.827 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=7ec07914-65bf-11ee-8db5-212e2dc638e7" will pass to kernel 2023-10-08T09:55:44.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T09:55:44.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:44.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.862 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:55:44.935 INFO:teuthology.orchestra.run.smithi097.stdout:client15546 2023-10-08T09:55:44.935 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:55:44.935 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:55:44.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:44.994 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:55:45.068 INFO:teuthology.orchestra.run.smithi097.stdout:client15546 2023-10-08T09:55:45.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:45.069 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:55:45.141 INFO:teuthology.orchestra.run.smithi097.stdout:client15546 2023-10-08T09:55:45.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:55:45.141 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7.client15546/status') 2023-10-08T09:55:45.221 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.15546 (3)192.168.0.1:0/807508162 2023-10-08T09:55:45.221 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2023-10-08T09:55:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2023-10-08T09:55:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2023-10-08T09:55:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 2.2879e-05 s, 3.0 MB/s 2023-10-08T09:55:45.223 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-08T09:55:45.234 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-08T09:55:45.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:45.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 -- 172.21.15.97:0/3550736802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08101120 msgr2=0x7f7f08103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/3550736802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08101120 0x7f7f08103510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009990 tx=0x7f7efc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 -- 172.21.15.97:0/3550736802 shutdown_connections 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/3550736802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08106380 0x7f7f081087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/3550736802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08101120 0x7f7f08103510 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/3550736802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 0x7f7f08105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 -- 172.21.15.97:0/3550736802 >> 172.21.15.97:0/3550736802 conn(0x7f7f080fab10 msgr2=0x7f7f080fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 -- 172.21.15.97:0/3550736802 shutdown_connections 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.477+0000 7f7f0d47a700 1 -- 172.21.15.97:0/3550736802 wait complete. 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 Processor -- start 2023-10-08T09:55:45.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 -- start start 2023-10-08T09:55:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08101120 0x7f7f08196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 0x7f7f08196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819e140 con 0x7f7f08106380 2023-10-08T09:55:45.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819e280 con 0x7f7f08101120 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f0d47a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819e410 con 0x7f7f08103a50 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56820/0 (socket says 172.21.15.97:56820) 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f077fe700 1 -- 172.21.15.97:0/2706107128 learned_addr learned my addr 172.21.15.97:0/2706107128 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f06ffd700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08101120 0x7f7f08196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.478+0000 7f7f067fc700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 0x7f7f08196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 -- 172.21.15.97:0/2706107128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 msgr2=0x7f7f08196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 0x7f7f08196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 -- 172.21.15.97:0/2706107128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08101120 msgr2=0x7f7f08196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08101120 0x7f7f08196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 -- 172.21.15.97:0/2706107128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0095f0 con 0x7f7f08106380 2023-10-08T09:55:45.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f077fe700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007ee0 tx=0x7f7f0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00017070 con 0x7f7f08106380 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0000f040 con 0x7f7f08106380 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0819e6f0 con 0x7f7f08106380 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00005580 con 0x7f7f08106380 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.479+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0819ebb0 con 0x7f7f08106380 2023-10-08T09:55:45.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.480+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080fc690 con 0x7f7f08106380 2023-10-08T09:55:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.481+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7f0000db20 con 0x7f7f08106380 2023-10-08T09:55:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.482+0000 7f7eeffff700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 0x7f7ef00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.482+0000 7f7f06ffd700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 0x7f7ef00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.482+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7f000cc770 con 0x7f7f08106380 2023-10-08T09:55:45.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.482+0000 7f7f06ffd700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 0x7f7ef00a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b5c0 tx=0x7f7efc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:45.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.486+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7f00095ac0 con 0x7f7f08106380 2023-10-08T09:55:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:45 smithi186 ceph-mon[104393]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.678+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f080678a0 con 0x7f7f08106380 2023-10-08T09:55:45.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.679+0000 7f7eeffff700 1 -- 172.21.15.97:0/2706107128 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24499 (secure 0 0 0) 0x7f7f000990e0 con 0x7f7f08106380 2023-10-08T09:55:45.682 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:45.682 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":87,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:54:47.232892+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:45.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.681+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 msgr2=0x7f7ef00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.681+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 0x7f7ef00a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b5c0 tx=0x7f7efc005fd0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 msgr2=0x7f7f0819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007ee0 tx=0x7f7f0000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 shutdown_connections 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7ef00a1480 0x7f7ef00a3940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f08106380 0x7f7f0819db70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f08101120 0x7f7f08196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 --2- 172.21.15.97:0/2706107128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f08103a50 0x7f7f08196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 >> 172.21.15.97:0/2706107128 conn(0x7f7f080fab10 msgr2=0x7f7f08101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:45.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 shutdown_connections 2023-10-08T09:55:45.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.682+0000 7f7f0d47a700 1 -- 172.21.15.97:0/2706107128 wait complete. 2023-10-08T09:55:45.695 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:45 smithi097 ceph-mon[99491]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:45.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:45.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:45.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:45 smithi178 ceph-mon[103156]: pgmap v316: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:55:45.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 -- 172.21.15.97:0/3970304587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed40ff8a0 msgr2=0x7efed40ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 --2- 172.21.15.97:0/3970304587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed40ff8a0 0x7efed40ffd20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 -- 172.21.15.97:0/3970304587 shutdown_connections 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 --2- 172.21.15.97:0/3970304587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed4100260 0x7efed4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 --2- 172.21.15.97:0/3970304587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100c50 0x7efed4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 --2- 172.21.15.97:0/3970304587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed40ff8a0 0x7efed40ffd20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 -- 172.21.15.97:0/3970304587 >> 172.21.15.97:0/3970304587 conn(0x7efed40fa390 msgr2=0x7efed40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 -- 172.21.15.97:0/3970304587 shutdown_connections 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.819+0000 7efeda587700 1 -- 172.21.15.97:0/3970304587 wait complete. 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 Processor -- start 2023-10-08T09:55:45.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 -- start start 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 0x7efed40ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100260 0x7efed419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419e150 con 0x7efed4100c50 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419e2c0 con 0x7efed40ff8a0 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efeda587700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419e480 con 0x7efed4100260 2023-10-08T09:55:45.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56834/0 (socket says 172.21.15.97:56834) 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 -- 172.21.15.97:0/250182092 learned_addr learned my addr 172.21.15.97:0/250182092 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed3fff700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 0x7efed40ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed37fe700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100260 0x7efed419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 -- 172.21.15.97:0/250182092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100260 msgr2=0x7efed419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100260 0x7efed419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 -- 172.21.15.97:0/250182092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 msgr2=0x7efed40ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.820+0000 7efed8b24700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 0x7efed40ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed8b24700 1 -- 172.21.15.97:0/250182092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80095f0 con 0x7efed4100c50 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed3fff700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 0x7efed40ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed8b24700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efecc007d70 tx=0x7efecc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:45.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00ad40 con 0x7efed4100c50 2023-10-08T09:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efecc0128e0 con 0x7efed4100c50 2023-10-08T09:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed419e760 con 0x7efed4100c50 2023-10-08T09:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc011780 con 0x7efed4100c50 2023-10-08T09:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.821+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed419ec70 con 0x7efed4100c50 2023-10-08T09:55:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.822+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed4106a70 con 0x7efed4100c50 2023-10-08T09:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.823+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efecc012a50 con 0x7efed4100c50 2023-10-08T09:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.823+0000 7efed17fa700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 0x7efebc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.823+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efecc0cd340 con 0x7efed4100c50 2023-10-08T09:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.823+0000 7efed3fff700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 0x7efebc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:45.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.824+0000 7efed3fff700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 0x7efebc0a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efec4009cb0 tx=0x7efec4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:45.828+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efecc096690 con 0x7efed4100c50 2023-10-08T09:55:46.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.021+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efed4106450 con 0x7efed4100c50 2023-10-08T09:55:46.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.021+0000 7efed17fa700 1 -- 172.21.15.97:0/250182092 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7efecc099cb0 con 0x7efed4100c50 2023-10-08T09:55:46.024 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:46.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 msgr2=0x7efebc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 0x7efebc0a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efec4009cb0 tx=0x7efec4009260 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 msgr2=0x7efed419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efecc007d70 tx=0x7efecc00a340 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 shutdown_connections 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7efebc0a1480 0x7efebc0a3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed4100c50 0x7efed419d9b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efed40ff8a0 0x7efed40ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 --2- 172.21.15.97:0/250182092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efed4100260 0x7efed419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 >> 172.21.15.97:0/250182092 conn(0x7efed40fa390 msgr2=0x7efed4103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 shutdown_connections 2023-10-08T09:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.024+0000 7efeda587700 1 -- 172.21.15.97:0/250182092 wait complete. 2023-10-08T09:55:46.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 -- 172.21.15.97:0/1631167468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e600ff5c0 msgr2=0x7f6e600ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/1631167468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e600ff5c0 0x7f6e600ff9e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6e540099c0 tx=0x7f6e54009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 -- 172.21.15.97:0/1631167468 shutdown_connections 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/1631167468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e60101150 0x7f6e60107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/1631167468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e600ff5c0 0x7f6e600ff9e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/1631167468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 0x7f6e60100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 -- 172.21.15.97:0/1631167468 >> 172.21.15.97:0/1631167468 conn(0x7f6e600fab30 msgr2=0x7f6e600fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 -- 172.21.15.97:0/1631167468 shutdown_connections 2023-10-08T09:55:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.161+0000 7f6e66cfb700 1 -- 172.21.15.97:0/1631167468 wait complete. 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 Processor -- start 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 -- start start 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e600ff5c0 0x7f6e60195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 0x7f6e60195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6019d080 con 0x7f6e60101150 2023-10-08T09:55:46.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6019d1c0 con 0x7f6e601007b0 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e66cfb700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6019d350 con 0x7f6e600ff5c0 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56848/0 (socket says 172.21.15.97:56848) 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 -- 172.21.15.97:0/3652839430 learned_addr learned my addr 172.21.15.97:0/3652839430 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e64a97700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e600ff5c0 0x7f6e60195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e5ffff700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 0x7f6e60195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 -- 172.21.15.97:0/3652839430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e600ff5c0 msgr2=0x7f6e60195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e600ff5c0 0x7f6e60195550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 -- 172.21.15.97:0/3652839430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 msgr2=0x7f6e60195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 0x7f6e60195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.162+0000 7f6e65298700 1 -- 172.21.15.97:0/3652839430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540095f0 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e65298700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ada0 tx=0x7f6e5800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800f070 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e58007b30 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6019d630 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58005280 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.163+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6019daf0 con 0x7f6e60101150 2023-10-08T09:55:46.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.164+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e60106a90 con 0x7f6e60101150 2023-10-08T09:55:46.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.165+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6e5800c8d0 con 0x7f6e60101150 2023-10-08T09:55:46.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.165+0000 7f6e5dffb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 0x7f6e500a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.166+0000 7f6e64a97700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 0x7f6e500a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.166+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6e580cc580 con 0x7f6e60101150 2023-10-08T09:55:46.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.166+0000 7f6e64a97700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 0x7f6e500a3930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e54003820 tx=0x7f6e54005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.170+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e58099390 con 0x7f6e60101150 2023-10-08T09:55:46.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.365+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e60106470 con 0x7f6e60101150 2023-10-08T09:55:46.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.366+0000 7f6e5dffb700 1 -- 172.21.15.97:0/3652839430 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e58098f20 con 0x7f6e60101150 2023-10-08T09:55:46.368 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 msgr2=0x7f6e500a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 0x7f6e500a3930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e54003820 tx=0x7f6e54005920 comp rx=0 tx=0).stop 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 msgr2=0x7f6e6019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ada0 tx=0x7f6e5800c330 comp rx=0 tx=0).stop 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 shutdown_connections 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6e500a1470 0x7f6e500a3930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e60101150 0x7f6e6019cab0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e600ff5c0 0x7f6e60195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 --2- 172.21.15.97:0/3652839430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e601007b0 0x7f6e60195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 >> 172.21.15.97:0/3652839430 conn(0x7f6e600fab30 msgr2=0x7f6e60103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 shutdown_connections 2023-10-08T09:55:46.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.368+0000 7f6e66cfb700 1 -- 172.21.15.97:0/3652839430 wait complete. 2023-10-08T09:55:46.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:46 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2706107128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:46.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:46 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/250182092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:46.381 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:46 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-08T09:55:46.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 -- 172.21.15.97:0/980984319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 msgr2=0x7fb2640ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 --2- 172.21.15.97:0/980984319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb2640ffac0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb2580099c0 tx=0x7fb258009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 -- 172.21.15.97:0/980984319 shutdown_connections 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 --2- 172.21.15.97:0/980984319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb264107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 --2- 172.21.15.97:0/980984319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb2640ffac0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 --2- 172.21.15.97:0/980984319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 0x7fb264100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 -- 172.21.15.97:0/980984319 >> 172.21.15.97:0/980984319 conn(0x7fb2640fac10 msgr2=0x7fb2640fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 -- 172.21.15.97:0/980984319 shutdown_connections 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.504+0000 7fb26a73c700 1 -- 172.21.15.97:0/980984319 wait complete. 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 Processor -- start 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 -- start start 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 0x7fb26419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb26419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26419e280 con 0x7fb2640ff6a0 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26419e3f0 con 0x7fb264100890 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb26a73c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26419e5b0 con 0x7fb264101230 2023-10-08T09:55:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56858/0 (socket says 172.21.15.97:56858) 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 -- 172.21.15.97:0/3443277586 learned_addr learned my addr 172.21.15.97:0/3443277586 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb2637fe700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 0x7fb26419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb268cd9700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb26419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 -- 172.21.15.97:0/3443277586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 msgr2=0x7fb26419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb26419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 -- 172.21.15.97:0/3443277586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 msgr2=0x7fb26419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 0x7fb26419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb263fff700 1 -- 172.21.15.97:0/3443277586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580095f0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.505+0000 7fb268cd9700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb26419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb263fff700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb258009fd0 tx=0x7fb258005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25801d070 con 0x7fb2640ff6a0 2023-10-08T09:55:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2580059a0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26419e7d0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580173f0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.506+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26419ecd0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.507+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb264106b40 con 0x7fb2640ff6a0 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.508+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb2580226e0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.508+0000 7fb2617fa700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 0x7fb24c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.508+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb2580ce820 con 0x7fb2640ff6a0 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.508+0000 7fb2637fe700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 0x7fb24c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.509+0000 7fb2637fe700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 0x7fb24c0a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb264068910 tx=0x7fb254009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.512+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb258097be0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:46 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2706107128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:46 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/250182092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:46.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:46 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-08T09:55:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:46 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2706107128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:46 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/250182092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:46.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:46 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-10-08T09:55:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.704+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb26410afd0 con 0x7fb2640ff6a0 2023-10-08T09:55:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.704+0000 7fb2617fa700 1 -- 172.21.15.97:0/3443277586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb25809b200 con 0x7fb2640ff6a0 2023-10-08T09:55:46.706 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 msgr2=0x7fb24c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 0x7fb24c0a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb264068910 tx=0x7fb254009260 comp rx=0 tx=0).stop 2023-10-08T09:55:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 msgr2=0x7fb264068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb258009fd0 tx=0x7fb258005f10 comp rx=0 tx=0).stop 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 shutdown_connections 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb24c0a1500 0x7fb24c0a39c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb264101230 0x7fb26419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2640ff6a0 0x7fb264068370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 --2- 172.21.15.97:0/3443277586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb264100890 0x7fb26419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 >> 172.21.15.97:0/3443277586 conn(0x7fb2640fac10 msgr2=0x7fb264103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 shutdown_connections 2023-10-08T09:55:46.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.706+0000 7fb26a73c700 1 -- 172.21.15.97:0/3443277586 wait complete. 2023-10-08T09:55:46.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.844+0000 7fb1b395f700 1 -- 172.21.15.97:0/774677339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1007c0 msgr2=0x7fb1ac100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.844+0000 7fb1b395f700 1 --2- 172.21.15.97:0/774677339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac100be0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0099c0 tx=0x7fb19c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- 172.21.15.97:0/774677339 shutdown_connections 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- 172.21.15.97:0/774677339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac102350 0x7fb1ac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- 172.21.15.97:0/774677339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac100be0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- 172.21.15.97:0/774677339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- 172.21.15.97:0/774677339 >> 172.21.15.97:0/774677339 conn(0x7fb1ac0fbd30 msgr2=0x7fb1ac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- 172.21.15.97:0/774677339 shutdown_connections 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- 172.21.15.97:0/774677339 wait complete. 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 Processor -- start 2023-10-08T09:55:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- start start 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac102350 0x7fb1ac19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19e210 con 0x7fb1ac1019b0 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19e350 con 0x7fb1ac102350 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b395f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19e4e0 con 0x7fb1ac1007c0 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.845+0000 7fb1b0efa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56874/0 (socket says 172.21.15.97:56874) 2023-10-08T09:55:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 -- 172.21.15.97:0/1763245870 learned_addr learned my addr 172.21.15.97:0/1763245870 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b16fb700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b1efc700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac102350 0x7fb1ac19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 -- 172.21.15.97:0/1763245870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac1007c0 msgr2=0x7fb1ac1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 -- 172.21.15.97:0/1763245870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac102350 msgr2=0x7fb1ac19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac102350 0x7fb1ac19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 -- 172.21.15.97:0/1763245870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0095f0 con 0x7fb1ac1019b0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b0efa700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c920 tx=0x7fb1a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8007b00 con 0x7fb1ac1019b0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1a800ce70 con 0x7fb1ac1019b0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac19e7c0 con 0x7fb1ac1019b0 2023-10-08T09:55:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80054e0 con 0x7fb1ac1019b0 2023-10-08T09:55:46.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.846+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac19ecd0 con 0x7fb1ac1019b0 2023-10-08T09:55:46.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.847+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac107b70 con 0x7fb1ac1019b0 2023-10-08T09:55:46.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.848+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb1a8004750 con 0x7fb1ac1019b0 2023-10-08T09:55:46.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.849+0000 7fb1a27fc700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 0x7fb1980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:46.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.849+0000 7fb1b16fb700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 0x7fb1980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:46.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.849+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb1a80ccbf0 con 0x7fb1ac1019b0 2023-10-08T09:55:46.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.849+0000 7fb1b16fb700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 0x7fb1980a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c006010 tx=0x7fb19c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:46.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:46.853+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1a8095f20 con 0x7fb1ac1019b0 2023-10-08T09:55:47.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.048+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb1ac10c250 con 0x7fb1ac1019b0 2023-10-08T09:55:47.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.048+0000 7fb1a27fc700 1 -- 172.21.15.97:0/1763245870 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb1a801d0a0 con 0x7fb1ac1019b0 2023-10-08T09:55:47.050 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 msgr2=0x7fb1980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 0x7fb1980a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c006010 tx=0x7fb19c00b540 comp rx=0 tx=0).stop 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 msgr2=0x7fb1ac196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c920 tx=0x7fb1a800cc30 comp rx=0 tx=0).stop 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 shutdown_connections 2023-10-08T09:55:47.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb1980a1480 0x7fb1980a3940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1ac102350 0x7fb1ac19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1ac1007c0 0x7fb1ac1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 --2- 172.21.15.97:0/1763245870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac1019b0 0x7fb1ac196c20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 >> 172.21.15.97:0/1763245870 conn(0x7fb1ac0fbd30 msgr2=0x7fb1ac104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 shutdown_connections 2023-10-08T09:55:47.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.050+0000 7fb1b395f700 1 -- 172.21.15.97:0/1763245870 wait complete. 2023-10-08T09:55:47.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:47.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 -- 172.21.15.97:0/2167556468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 msgr2=0x7f6dc8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/2167556468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8100c10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6db80099c0 tx=0x7f6db8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 -- 172.21.15.97:0/2167556468 shutdown_connections 2023-10-08T09:55:47.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/2167556468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 0x7f6dc8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/2167556468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc80ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/2167556468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8100c10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 -- 172.21.15.97:0/2167556468 >> 172.21.15.97:0/2167556468 conn(0x7f6dc80fab30 msgr2=0x7f6dc80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 -- 172.21.15.97:0/2167556468 shutdown_connections 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.184+0000 7f6dd06a0700 1 -- 172.21.15.97:0/2167556468 wait complete. 2023-10-08T09:55:47.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 Processor -- start 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 -- start start 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc8195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 0x7f6dc819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc819d030 con 0x7f6dc81007b0 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc819d170 con 0x7f6dc80ff5c0 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dd06a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc819d300 con 0x7f6dc8101150 2023-10-08T09:55:47.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dcdc3b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dcdc3b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56888/0 (socket says 172.21.15.97:56888) 2023-10-08T09:55:47.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dcdc3b700 1 -- 172.21.15.97:0/1719261928 learned_addr learned my addr 172.21.15.97:0/1719261928 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:47.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.185+0000 7f6dcec3d700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 0x7f6dc819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dce43c700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc8195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 -- 172.21.15.97:0/1719261928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 msgr2=0x7f6dc819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 0x7f6dc819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 -- 172.21.15.97:0/1719261928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 msgr2=0x7f6dc8195500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc8195500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 -- 172.21.15.97:0/1719261928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80095f0 con 0x7f6dc81007b0 2023-10-08T09:55:47.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dce43c700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc8195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:47.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dcdc3b700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6db8003730 tx=0x7f6db801a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db801f070 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc8068a70 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db80059a0 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8004a90 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.186+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc8068ec0 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.187+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc8066bc0 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.188+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6db8023070 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.188+0000 7f6dbf7fe700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 0x7f6db40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.189+0000 7f6dce43c700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 0x7f6db40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.189+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6db80d0120 con 0x7f6dc81007b0 2023-10-08T09:55:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.189+0000 7f6dce43c700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 0x7f6db40a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8100610 tx=0x7f6dc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.192+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6db80994e0 con 0x7f6dc81007b0 2023-10-08T09:55:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.392+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dc810ae00 con 0x7f6dc81007b0 2023-10-08T09:55:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.393+0000 7f6dbf7fe700 1 -- 172.21.15.97:0/1719261928 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f6db809cb00 con 0x7f6dc81007b0 2023-10-08T09:55:47.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:47.395 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 msgr2=0x7f6db40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 0x7f6db40a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8100610 tx=0x7f6dc4009260 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 msgr2=0x7f6dc8195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6db8003730 tx=0x7f6db801a820 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 shutdown_connections 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6db40a1480 0x7f6db40a3940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dc8101150 0x7f6dc819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dc80ff5c0 0x7f6dc8195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 --2- 172.21.15.97:0/1719261928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc81007b0 0x7f6dc8195a40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 >> 172.21.15.97:0/1719261928 conn(0x7f6dc80fab30 msgr2=0x7f6dc8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.395+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 shutdown_connections 2023-10-08T09:55:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.396+0000 7f6dd06a0700 1 -- 172.21.15.97:0/1719261928 wait complete. 2023-10-08T09:55:47.408 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:47.408 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-08T09:55:47.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:55:47.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3652839430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3443277586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.409 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:47 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1763245870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.411 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:47.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-08T09:55:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:55:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3652839430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3443277586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:47 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1763245870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- 172.21.15.97:0/1641275693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74102280 msgr2=0x7faf74108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/1641275693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74102280 0x7faf74108b30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b0d0 tx=0x7faf6c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- 172.21.15.97:0/1641275693 shutdown_connections 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/1641275693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74102280 0x7faf74108b30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/1641275693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf741006f0 0x7faf74100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/1641275693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741018e0 0x7faf74101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- 172.21.15.97:0/1641275693 >> 172.21.15.97:0/1641275693 conn(0x7faf740fbc60 msgr2=0x7faf740fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- 172.21.15.97:0/1641275693 shutdown_connections 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- 172.21.15.97:0/1641275693 wait complete. 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 Processor -- start 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 -- start start 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 0x7faf74196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.534+0000 7faf7c5ac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74102280 0x7faf7419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419e2c0 con 0x7faf741018e0 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419e400 con 0x7faf741006f0 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419e590 con 0x7faf74102280 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56902/0 (socket says 172.21.15.97:56902) 2023-10-08T09:55:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 -- 172.21.15.97:0/3859204334 learned_addr learned my addr 172.21.15.97:0/3859204334 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7ab49700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74102280 0x7faf7419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7a348700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 0x7faf74196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 -- 172.21.15.97:0/3859204334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74102280 msgr2=0x7faf7419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74102280 0x7faf7419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 -- 172.21.15.97:0/3859204334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 msgr2=0x7faf74196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 0x7faf74196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 -- 172.21.15.97:0/3859204334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009d00 con 0x7faf741018e0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7a348700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 0x7faf74196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf79b47700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b530 tx=0x7faf6400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64011830 con 0x7faf741018e0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7419e870 con 0x7faf741018e0 2023-10-08T09:55:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf64011e50 con 0x7faf741018e0 2023-10-08T09:55:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400f2e0 con 0x7faf741018e0 2023-10-08T09:55:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.535+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7419edd0 con 0x7faf741018e0 2023-10-08T09:55:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.536+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf74107aa0 con 0x7faf741018e0 2023-10-08T09:55:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.537+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faf64010380 con 0x7faf741018e0 2023-10-08T09:55:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.538+0000 7faf6b7fe700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 0x7faf600a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.538+0000 7faf7a348700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 0x7faf600a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.538+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faf640cd150 con 0x7faf741018e0 2023-10-08T09:55:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.538+0000 7faf7a348700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 0x7faf600a3930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf70005950 tx=0x7faf700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.541+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faf64096480 con 0x7faf741018e0 2023-10-08T09:55:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: pgmap v317: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-08T09:55:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-10-08T09:55:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:55:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3652839430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3443277586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:47 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1763245870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.732+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf7410bf80 con 0x7faf741018e0 2023-10-08T09:55:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.732+0000 7faf6b7fe700 1 -- 172.21.15.97:0/3859204334 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf64099aa0 con 0x7faf741018e0 2023-10-08T09:55:47.735 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:47.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 msgr2=0x7faf600a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 0x7faf600a3930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf70005950 tx=0x7faf700058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 msgr2=0x7faf74196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b530 tx=0x7faf6400b840 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 shutdown_connections 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1470 0x7faf600a3930 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74102280 0x7faf7419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf741006f0 0x7faf74196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 --2- 172.21.15.97:0/3859204334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741018e0 0x7faf74196cd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.734+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 >> 172.21.15.97:0/3859204334 conn(0x7faf740fbc60 msgr2=0x7faf74104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.735+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 shutdown_connections 2023-10-08T09:55:47.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.735+0000 7faf7c5ac700 1 -- 172.21.15.97:0/3859204334 wait complete. 2023-10-08T09:55:47.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 -- 172.21.15.97:0/1174276605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4100890 msgr2=0x7f9fb4100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 --2- 172.21.15.97:0/1174276605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4100890 0x7f9fb4100cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00099c0 tx=0x7f9fb0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 -- 172.21.15.97:0/1174276605 shutdown_connections 2023-10-08T09:55:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 --2- 172.21.15.97:0/1174276605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4101230 0x7f9fb4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 --2- 172.21.15.97:0/1174276605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb40ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 --2- 172.21.15.97:0/1174276605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4100890 0x7f9fb4100cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 -- 172.21.15.97:0/1174276605 >> 172.21.15.97:0/1174276605 conn(0x7f9fb40fac10 msgr2=0x7f9fb40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 -- 172.21.15.97:0/1174276605 shutdown_connections 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.871+0000 7f9fbb381700 1 -- 172.21.15.97:0/1174276605 wait complete. 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 Processor -- start 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 -- start start 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4100890 0x7f9fb419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 0x7f9fb419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419e160 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419e2d0 con 0x7f9fb4100890 2023-10-08T09:55:47.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fbb381700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb419e490 con 0x7f9fb4101230 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56914/0 (socket says 172.21.15.97:56914) 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 -- 172.21.15.97:0/3579139852 learned_addr learned my addr 172.21.15.97:0/3579139852 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb991e700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 0x7f9fb419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb891c700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4100890 0x7f9fb419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 -- 172.21.15.97:0/3579139852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 msgr2=0x7f9fb419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 0x7f9fb419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.872+0000 7f9fb911d700 1 -- 172.21.15.97:0/3579139852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4100890 msgr2=0x7f9fb419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fb911d700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4100890 0x7f9fb419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fb911d700 1 -- 172.21.15.97:0/3579139852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00095f0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fb991e700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 0x7f9fb419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:47.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fb911d700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d980 tx=0x7f9fa400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400de60 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa40044e0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4003be0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb419e770 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.873+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb419ec50 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.874+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb4106aa0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.875+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9fa4009bb0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.875+0000 7f9faa7fc700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 0x7f9fa00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.875+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9fa40cd0f0 con 0x7f9fb40ff6a0 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.875+0000 7f9fb891c700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 0x7f9fa00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:47.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.876+0000 7f9fb891c700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 0x7f9fa00a39c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:47.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:47.879+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fa4096440 con 0x7f9fb40ff6a0 2023-10-08T09:55:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.068+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fb410aeb0 con 0x7f9fb40ff6a0 2023-10-08T09:55:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.069+0000 7f9faa7fc700 1 -- 172.21.15.97:0/3579139852 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fa4009940 con 0x7f9fb40ff6a0 2023-10-08T09:55:48.071 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 msgr2=0x7f9fa00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 0x7f9fa00a39c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0005fd0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 msgr2=0x7f9fb4068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d980 tx=0x7f9fa400dd40 comp rx=0 tx=0).stop 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 shutdown_connections 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9fa00a1500 0x7f9fa00a39c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fb4101230 0x7f9fb419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb40ff6a0 0x7f9fb4068350 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 --2- 172.21.15.97:0/3579139852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fb4100890 0x7f9fb419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 >> 172.21.15.97:0/3579139852 conn(0x7f9fb40fac10 msgr2=0x7f9fb4103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 shutdown_connections 2023-10-08T09:55:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.071+0000 7f9fbb381700 1 -- 172.21.15.97:0/3579139852 wait complete. 2023-10-08T09:55:48.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- 172.21.15.97:0/2850621838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30280ff6a0 msgr2=0x7f30280ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2850621838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30280ff6a0 0x7f30280ffac0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f301c0099c0 tx=0x7f301c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- 172.21.15.97:0/2850621838 shutdown_connections 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2850621838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028101230 0x7f3028107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2850621838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30280ff6a0 0x7f30280ffac0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2850621838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3028100890 0x7f3028100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- 172.21.15.97:0/2850621838 >> 172.21.15.97:0/2850621838 conn(0x7f30280fac10 msgr2=0x7f30280fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- 172.21.15.97:0/2850621838 shutdown_connections 2023-10-08T09:55:48.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- 172.21.15.97:0/2850621838 wait complete. 2023-10-08T09:55:48.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 Processor -- start 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.208+0000 7f302dc0e700 1 -- start start 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 0x7f30280683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 0x7f302819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819e1a0 con 0x7f3028101230 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819e310 con 0x7f30280ff6a0 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302819e4d0 con 0x7f3028100890 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56932/0 (socket says 172.21.15.97:56932) 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 -- 172.21.15.97:0/2295553847 learned_addr learned my addr 172.21.15.97:0/2295553847 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f30277fe700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 0x7f30280683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3026ffd700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 0x7f302819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 -- 172.21.15.97:0/2295553847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 msgr2=0x7f302819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 0x7f302819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 -- 172.21.15.97:0/2295553847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 msgr2=0x7f30280683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 0x7f30280683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 -- 172.21.15.97:0/2295553847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0095f0 con 0x7f3028101230 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3026ffd700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 0x7f302819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f30277fe700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 0x7f30280683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3027fff700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3020007ee0 tx=0x7f302000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020017070 con 0x7f3028101230 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f302000f040 con 0x7f3028101230 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302819e7b0 con 0x7f3028101230 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020005740 con 0x7f3028101230 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.209+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302819ec70 con 0x7f3028101230 2023-10-08T09:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.211+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3028106ad0 con 0x7f3028101230 2023-10-08T09:55:48.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.212+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f30200058a0 con 0x7f3028101230 2023-10-08T09:55:48.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.212+0000 7f3024ff9700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 0x7f30100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.213+0000 7f30277fe700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 0x7f30100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.213+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f30200cc900 con 0x7f3028101230 2023-10-08T09:55:48.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.213+0000 7f30277fe700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 0x7f30100a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f301c006010 tx=0x7f301c009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.217+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3020095c30 con 0x7f3028101230 2023-10-08T09:55:48.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.408+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30281064b0 con 0x7f3028101230 2023-10-08T09:55:48.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.408+0000 7f3024ff9700 1 -- 172.21.15.97:0/2295553847 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3020099250 con 0x7f3028101230 2023-10-08T09:55:48.410 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:48.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 msgr2=0x7f30100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 0x7f30100a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f301c006010 tx=0x7f301c009530 comp rx=0 tx=0).stop 2023-10-08T09:55:48.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 msgr2=0x7f302819da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3020007ee0 tx=0x7f302000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 shutdown_connections 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f30100a1480 0x7f30100a3940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3028101230 0x7f302819da00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30280ff6a0 0x7f30280683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 --2- 172.21.15.97:0/2295553847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3028100890 0x7f302819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 >> 172.21.15.97:0/2295553847 conn(0x7f30280fac10 msgr2=0x7f3028103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 shutdown_connections 2023-10-08T09:55:48.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.410+0000 7f302dc0e700 1 -- 172.21.15.97:0/2295553847 wait complete. 2023-10-08T09:55:48.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:48 smithi097 ceph-mon[99491]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:55:48.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:48 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1719261928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:48.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:48 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3859204334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:48 smithi097 ceph-mon[99491]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:48.424 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:48 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3579139852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:48 smithi186 ceph-mon[104393]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:55:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:48 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1719261928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:48 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3859204334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:48 smithi186 ceph-mon[104393]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:48.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:48 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3579139852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- 172.21.15.97:0/979438782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 msgr2=0x7ff17c0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 --2- 172.21.15.97:0/979438782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c0ffac0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0099c0 tx=0x7ff16c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- 172.21.15.97:0/979438782 shutdown_connections 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 --2- 172.21.15.97:0/979438782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c101230 0x7ff17c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 --2- 172.21.15.97:0/979438782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c0ffac0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 --2- 172.21.15.97:0/979438782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c100890 0x7ff17c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- 172.21.15.97:0/979438782 >> 172.21.15.97:0/979438782 conn(0x7ff17c0fac10 msgr2=0x7ff17c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- 172.21.15.97:0/979438782 shutdown_connections 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- 172.21.15.97:0/979438782 wait complete. 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 Processor -- start 2023-10-08T09:55:48.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.547+0000 7ff183f71700 1 -- start start 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 0x7ff17c195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 0x7ff17c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19d1d0 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19d310 con 0x7ff17c101230 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c19d4a0 con 0x7ff17c100890 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56942/0 (socket says 172.21.15.97:56942) 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 -- 172.21.15.97:0/2478590539 learned_addr learned my addr 172.21.15.97:0/2478590539 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:48.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff18250e700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 0x7ff17c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 -- 172.21.15.97:0/2478590539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 msgr2=0x7ff17c195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff18150c700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 0x7ff17c195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 0x7ff17c195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 -- 172.21.15.97:0/2478590539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 msgr2=0x7ff17c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 0x7ff17c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 -- 172.21.15.97:0/2478590539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0095f0 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff18250e700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 0x7ff17c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff18150c700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 0x7ff17c195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff181d0d700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006010 tx=0x7ff16c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c01d070 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16c004160 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c19d720 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0053b0 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.548+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c19db40 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.549+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c106b10 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.550+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff16c005510 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.551+0000 7ff172ffd700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 0x7ff1680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.551+0000 7ff18150c700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 0x7ff1680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.551+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff16c0ce2d0 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.552+0000 7ff18150c700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 0x7ff1680a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff178005fd0 tx=0x7ff178005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.555+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff16c05b100 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:48 smithi178 ceph-mon[103156]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:55:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:48 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1719261928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:48 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3859204334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:48 smithi178 ceph-mon[103156]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:48 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3579139852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:48.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.748+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff17c10af00 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.748+0000 7ff172ffd700 1 -- 172.21.15.97:0/2478590539 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff16c097730 con 0x7ff17c0ff6a0 2023-10-08T09:55:48.750 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 msgr2=0x7ff1680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 0x7ff1680a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff178005fd0 tx=0x7ff178005dc0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 msgr2=0x7ff17c1956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006010 tx=0x7ff16c005af0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 shutdown_connections 2023-10-08T09:55:48.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1680a1480 0x7ff1680a3940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17c101230 0x7ff17c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17c0ff6a0 0x7ff17c1956a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 --2- 172.21.15.97:0/2478590539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff17c100890 0x7ff17c195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 >> 172.21.15.97:0/2478590539 conn(0x7ff17c0fac10 msgr2=0x7ff17c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 shutdown_connections 2023-10-08T09:55:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.750+0000 7ff183f71700 1 -- 172.21.15.97:0/2478590539 wait complete. 2023-10-08T09:55:48.765 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-08T09:55:48.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:48.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 -- 172.21.15.97:0/2977009471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 msgr2=0x7f41c4108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 --2- 172.21.15.97:0/2977009471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4108b10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b0d0 tx=0x7f41bc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 -- 172.21.15.97:0/2977009471 shutdown_connections 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 --2- 172.21.15.97:0/2977009471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4108b10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 --2- 172.21.15.97:0/2977009471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 0x7f41c4100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 --2- 172.21.15.97:0/2977009471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 -- 172.21.15.97:0/2977009471 >> 172.21.15.97:0/2977009471 conn(0x7f41c40fbc40 msgr2=0x7f41c40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 -- 172.21.15.97:0/2977009471 shutdown_connections 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.885+0000 7f41ca65b700 1 -- 172.21.15.97:0/2977009471 wait complete. 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 Processor -- start 2023-10-08T09:55:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 -- start start 2023-10-08T09:55:48.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 0x7f41c4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40738c0 con 0x7f41c4102260 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4073a30 con 0x7f41c41018c0 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41ca65b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c41a0750 con 0x7f41c41006d0 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56960/0 (socket says 172.21.15.97:56960) 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 -- 172.21.15.97:0/1178966003 learned_addr learned my addr 172.21.15.97:0/1178966003 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c37fe700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c3fff700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 0x7f41c4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 -- 172.21.15.97:0/1178966003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 msgr2=0x7f41c4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 0x7f41c4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 -- 172.21.15.97:0/1178966003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 msgr2=0x7f41c4072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c8bf8700 1 -- 172.21.15.97:0/1178966003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009d00 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.886+0000 7f41c37fe700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41c8bf8700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f41bc007bd0 tx=0x7f41bc0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00e040 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc012dc0 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c41a0980 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc01b450 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.887+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c41a0d50 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.888+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4107b00 con 0x7f41c4102260 2023-10-08T09:55:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.889+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f41bc01d610 con 0x7f41c4102260 2023-10-08T09:55:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.889+0000 7f41c17fa700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 0x7f41ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.889+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f41bc0cf3e0 con 0x7f41c4102260 2023-10-08T09:55:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.889+0000 7f41c3fff700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 0x7f41ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.890+0000 7f41c3fff700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 0x7f41ac0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f41c4101720 tx=0x7f41b400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:48.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:48.893+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41bc0987a0 con 0x7f41c4102260 2023-10-08T09:55:49.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.128+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41c4067450 con 0x7f41c4102260 2023-10-08T09:55:49.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.129+0000 7f41c17fa700 1 -- 172.21.15.97:0/1178966003 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f41bc09bdc0 con 0x7f41c4102260 2023-10-08T09:55:49.131 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:49.131 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 msgr2=0x7f41ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 0x7f41ac0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f41c4101720 tx=0x7f41b400b5b0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 msgr2=0x7f41c4073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f41bc007bd0 tx=0x7f41bc0094f0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 shutdown_connections 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f41ac0a1480 0x7f41ac0a3940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4102260 0x7f41c4073120 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41c41006d0 0x7f41c4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 --2- 172.21.15.97:0/1178966003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41c41018c0 0x7f41c4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 >> 172.21.15.97:0/1178966003 conn(0x7f41c40fbc40 msgr2=0x7f41c4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:49.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 shutdown_connections 2023-10-08T09:55:49.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.131+0000 7f41ca65b700 1 -- 172.21.15.97:0/1178966003 wait complete. 2023-10-08T09:55:49.134 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:49.145 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-08T09:55:49.145 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-08T09:55:49.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 -- 172.21.15.97:0/3044448373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 msgr2=0x7f9ac4105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/3044448373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4105e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009990 tx=0x7f9ab8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 -- 172.21.15.97:0/3044448373 shutdown_connections 2023-10-08T09:55:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/3044448373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4106380 0x7f9ac41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/3044448373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4101120 0x7f9ac4103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/3044448373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4105e40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 -- 172.21.15.97:0/3044448373 >> 172.21.15.97:0/3044448373 conn(0x7f9ac40fab10 msgr2=0x7f9ac40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 -- 172.21.15.97:0/3044448373 shutdown_connections 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.266+0000 7f9ac98c3700 1 -- 172.21.15.97:0/3044448373 wait complete. 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 Processor -- start 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 -- start start 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4106380 0x7f9ac419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419e1c0 con 0x7f9ac4101120 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419e300 con 0x7f9ac4106380 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac98c3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac419e490 con 0x7f9ac4103a50 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56970/0 (socket says 172.21.15.97:56970) 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac2ffd700 1 -- 172.21.15.97:0/1020379334 learned_addr learned my addr 172.21.15.97:0/1020379334 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.267+0000 7f9ac27fc700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac37fe700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4106380 0x7f9ac419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 -- 172.21.15.97:0/1020379334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 msgr2=0x7f9ac4196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 -- 172.21.15.97:0/1020379334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4106380 msgr2=0x7f9ac419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4106380 0x7f9ac419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 -- 172.21.15.97:0/1020379334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4009520 con 0x7f9ac4101120 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac27fc700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac2ffd700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400eb30 tx=0x7f9ab400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4009880 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ab40044e0 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80095f0 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4010430 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.268+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac419eb70 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.269+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40fc690 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.270+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9ab4003ee0 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.271+0000 7f9ac88c1700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 0x7f9aac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.271+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9ab4014070 con 0x7f9ac4101120 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.271+0000 7f9ac27fc700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 0x7f9aac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.271+0000 7f9ac27fc700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 0x7f9aac0a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b2e0 tx=0x7f9ab8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.274+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9ab40969b0 con 0x7f9ac4101120 2023-10-08T09:55:49.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:49 smithi097 ceph-mon[99491]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-08T09:55:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:49 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2295553847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:49 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2478590539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:49 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1178966003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.460+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ac4060c30 con 0x7f9aac0a1480 2023-10-08T09:55:49.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.473+0000 7f9ac88c1700 1 -- 172.21.15.97:0/1020379334 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427408 (secure 0 0 0) 0x7f9ac4060c30 con 0x7f9aac0a1480 2023-10-08T09:55:49.480 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 msgr2=0x7f9aac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 0x7f9aac0a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b2e0 tx=0x7f9ab8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 msgr2=0x7f9ac4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400eb30 tx=0x7f9ab400eef0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 shutdown_connections 2023-10-08T09:55:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9aac0a1480 0x7f9aac0a3940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ac4106380 0x7f9ac419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac4101120 0x7f9ac4196690 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 --2- 172.21.15.97:0/1020379334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ac4103a50 0x7f9ac4196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 >> 172.21.15.97:0/1020379334 conn(0x7f9ac40fab10 msgr2=0x7f9ac4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 shutdown_connections 2023-10-08T09:55:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.477+0000 7f9ac98c3700 1 -- 172.21.15.97:0/1020379334 wait complete. 2023-10-08T09:55:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:49.492 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2023-10-08T09:55:47.907130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":629,"num_read_kb":374,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337604,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604220,"statfs":{"total":1151940427776,"available":1151594721280,"internally_reserved":0,"allocated":13676544,"data_stored":7527424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002293"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243442+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:47.243442+0000","last_peered":"2023-10-08T09:54:47.243442+0000","last_clean":"2023-10-08T09:54:47.243442+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:47.243442+0000","last_undegraded":"2023-10-08T09:54:47.243442+0000","last_fullsized":"2023-10-08T09:54:47.243442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236793+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:47.236793+0000","last_peered":"2023-10-08T09:54:47.236793+0000","last_clean":"2023-10-08T09:54:47.236793+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:47.236793+0000","last_undegraded":"2023-10-08T09:54:47.236793+0000","last_fullsized":"2023-10-08T09:54:47.236793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.237042+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:47.237042+0000","last_peered":"2023-10-08T09:54:47.237042+0000","last_clean":"2023-10-08T09:54:47.237042+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:47.237042+0000","last_undegraded":"2023-10-08T09:54:47.237042+0000","last_fullsized":"2023-10-08T09:54:47.237042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235313+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:46.235313+0000","last_peered":"2023-10-08T09:54:46.235313+0000","last_clean":"2023-10-08T09:54:46.235313+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:46.235313+0000","last_undegraded":"2023-10-08T09:54:46.235313+0000","last_fullsized":"2023-10-08T09:54:46.235313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236665+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:47.236665+0000","last_peered":"2023-10-08T09:54:47.236665+0000","last_clean":"2023-10-08T09:54:47.236665+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:47.236665+0000","last_undegraded":"2023-10-08T09:54:47.236665+0000","last_fullsized":"2023-10-08T09:54:47.236665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236962+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:47.236962+0000","last_peered":"2023-10-08T09:54:47.236962+0000","last_clean":"2023-10-08T09:54:47.236962+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:47.236962+0000","last_undegraded":"2023-10-08T09:54:47.236962+0000","last_fullsized":"2023-10-08T09:54:47.236962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243351+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:47.243351+0000","last_peered":"2023-10-08T09:54:47.243351+0000","last_clean":"2023-10-08T09:54:47.243351+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:47.243351+0000","last_undegraded":"2023-10-08T09:54:47.243351+0000","last_fullsized":"2023-10-08T09:54:47.243351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242285+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:47.242285+0000","last_peered":"2023-10-08T09:54:47.242285+0000","last_clean":"2023-10-08T09:54:47.242285+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:47.242285+0000","last_undegraded":"2023-10-08T09:54:47.242285+0000","last_fullsized":"2023-10-08T09:54:47.242285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239781+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:46.239781+0000","last_peered":"2023-10-08T09:54:46.239781+0000","last_clean":"2023-10-08T09:54:46.239781+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:46.239781+0000","last_undegraded":"2023-10-08T09:54:46.239781+0000","last_fullsized":"2023-10-08T09:54:46.239781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372615+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:54:47.372615+0000","last_peered":"2023-10-08T09:54:47.372615+0000","last_clean":"2023-10-08T09:54:47.372615+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:54:47.372615+0000","last_undegraded":"2023-10-08T09:54:47.372615+0000","last_fullsized":"2023-10-08T09:54:47.372615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242460+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:47.242460+0000","last_peered":"2023-10-08T09:54:47.242460+0000","last_clean":"2023-10-08T09:54:47.242460+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:47.242460+0000","last_undegraded":"2023-10-08T09:54:47.242460+0000","last_fullsized":"2023-10-08T09:54:47.242460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235877+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:46.235877+0000","last_peered":"2023-10-08T09:54:46.235877+0000","last_clean":"2023-10-08T09:54:46.235877+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:46.235877+0000","last_undegraded":"2023-10-08T09:54:46.235877+0000","last_fullsized":"2023-10-08T09:54:46.235877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235918+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:46.235918+0000","last_peered":"2023-10-08T09:54:46.235918+0000","last_clean":"2023-10-08T09:54:46.235918+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:46.235918+0000","last_undegraded":"2023-10-08T09:54:46.235918+0000","last_fullsized":"2023-10-08T09:54:46.235918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236960+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:47.236960+0000","last_peered":"2023-10-08T09:54:47.236960+0000","last_clean":"2023-10-08T09:54:47.236960+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:47.236960+0000","last_undegraded":"2023-10-08T09:54:47.236960+0000","last_fullsized":"2023-10-08T09:54:47.236960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239642+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:46.239642+0000","last_peered":"2023-10-08T09:54:46.239642+0000","last_clean":"2023-10-08T09:54:46.239642+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:46.239642+0000","last_undegraded":"2023-10-08T09:54:46.239642+0000","last_fullsized":"2023-10-08T09:54:46.239642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.310191+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:43.310191+0000","last_peered":"2023-10-08T09:55:43.310191+0000","last_clean":"2023-10-08T09:55:43.310191+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:43.310191+0000","last_undegraded":"2023-10-08T09:55:43.310191+0000","last_fullsized":"2023-10-08T09:55:43.310191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239716+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:46.239716+0000","last_peered":"2023-10-08T09:54:46.239716+0000","last_clean":"2023-10-08T09:54:46.239716+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:46.239716+0000","last_undegraded":"2023-10-08T09:54:46.239716+0000","last_fullsized":"2023-10-08T09:54:46.239716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243249+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:47.243249+0000","last_peered":"2023-10-08T09:54:47.243249+0000","last_clean":"2023-10-08T09:54:47.243249+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:47.243249+0000","last_undegraded":"2023-10-08T09:54:47.243249+0000","last_fullsized":"2023-10-08T09:54:47.243249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235223+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:46.235223+0000","last_peered":"2023-10-08T09:54:46.235223+0000","last_clean":"2023-10-08T09:54:46.235223+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:46.235223+0000","last_undegraded":"2023-10-08T09:54:46.235223+0000","last_fullsized":"2023-10-08T09:54:46.235223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236698+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:47.236698+0000","last_peered":"2023-10-08T09:54:47.236698+0000","last_clean":"2023-10-08T09:54:47.236698+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:47.236698+0000","last_undegraded":"2023-10-08T09:54:47.236698+0000","last_fullsized":"2023-10-08T09:54:47.236698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242154+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:47.242154+0000","last_peered":"2023-10-08T09:54:47.242154+0000","last_clean":"2023-10-08T09:54:47.242154+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:47.242154+0000","last_undegraded":"2023-10-08T09:54:47.242154+0000","last_fullsized":"2023-10-08T09:54:47.242154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235537+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:46.235537+0000","last_peered":"2023-10-08T09:54:46.235537+0000","last_clean":"2023-10-08T09:54:46.235537+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:46.235537+0000","last_undegraded":"2023-10-08T09:54:46.235537+0000","last_fullsized":"2023-10-08T09:54:46.235537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.311163+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:44.311163+0000","last_peered":"2023-10-08T09:55:44.311163+0000","last_clean":"2023-10-08T09:55:44.311163+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:44.311163+0000","last_undegraded":"2023-10-08T09:55:44.311163+0000","last_fullsized":"2023-10-08T09:55:44.311163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.241714+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:47.241714+0000","last_peered":"2023-10-08T09:54:47.241714+0000","last_clean":"2023-10-08T09:54:47.241714+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:47.241714+0000","last_undegraded":"2023-10-08T09:54:47.241714+0000","last_fullsized":"2023-10-08T09:54:47.241714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236932+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:47.236932+0000","last_peered":"2023-10-08T09:54:47.236932+0000","last_clean":"2023-10-08T09:54:47.236932+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:47.236932+0000","last_undegraded":"2023-10-08T09:54:47.236932+0000","last_fullsized":"2023-10-08T09:54:47.236932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242039+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:47.242039+0000","last_peered":"2023-10-08T09:54:47.242039+0000","last_clean":"2023-10-08T09:54:47.242039+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:47.242039+0000","last_undegraded":"2023-10-08T09:54:47.242039+0000","last_fullsized":"2023-10-08T09:54:47.242039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":220,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:45.330927+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:45.330927+0000","last_peered":"2023-10-08T09:55:45.330927+0000","last_clean":"2023-10-08T09:55:45.330927+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:45.330927+0000","last_undegraded":"2023-10-08T09:55:45.330927+0000","last_fullsized":"2023-10-08T09:55:45.330927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":85,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240097+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:46.240097+0000","last_peered":"2023-10-08T09:54:46.240097+0000","last_clean":"2023-10-08T09:54:46.240097+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:46.240097+0000","last_undegraded":"2023-10-08T09:54:46.240097+0000","last_fullsized":"2023-10-08T09:54:46.240097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242355+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:47.242355+0000","last_peered":"2023-10-08T09:54:47.242355+0000","last_clean":"2023-10-08T09:54:47.242355+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:47.242355+0000","last_undegraded":"2023-10-08T09:54:47.242355+0000","last_fullsized":"2023-10-08T09:54:47.242355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242227+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:47.242227+0000","last_peered":"2023-10-08T09:54:47.242227+0000","last_clean":"2023-10-08T09:54:47.242227+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:47.242227+0000","last_undegraded":"2023-10-08T09:54:47.242227+0000","last_fullsized":"2023-10-08T09:54:47.242227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.309721+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:43.309721+0000","last_peered":"2023-10-08T09:55:43.309721+0000","last_clean":"2023-10-08T09:55:43.309721+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:43.309721+0000","last_undegraded":"2023-10-08T09:55:43.309721+0000","last_fullsized":"2023-10-08T09:55:43.309721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.268332+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.268332+0000","last_peered":"2023-10-08T09:54:47.268332+0000","last_clean":"2023-10-08T09:54:47.268332+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.268332+0000","last_undegraded":"2023-10-08T09:54:47.268332+0000","last_fullsized":"2023-10-08T09:54:47.268332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242632+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:47.242632+0000","last_peered":"2023-10-08T09:54:47.242632+0000","last_clean":"2023-10-08T09:54:47.242632+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:47.242632+0000","last_undegraded":"2023-10-08T09:54:47.242632+0000","last_fullsized":"2023-10-08T09:54:47.242632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242315+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:47.242315+0000","last_peered":"2023-10-08T09:54:47.242315+0000","last_clean":"2023-10-08T09:54:47.242315+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:47.242315+0000","last_undegraded":"2023-10-08T09:54:47.242315+0000","last_fullsized":"2023-10-08T09:54:47.242315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.310385+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:44.310385+0000","last_peered":"2023-10-08T09:55:44.310385+0000","last_clean":"2023-10-08T09:55:44.310385+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:44.310385+0000","last_undegraded":"2023-10-08T09:55:44.310385+0000","last_fullsized":"2023-10-08T09:55:44.310385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-10-08T09:53:56.877269+0000","last_change":"2023-10-08T09:55:46.965654+0000","last_active":"2023-10-08T09:53:56.178636+0000","last_peered":"2023-10-08T09:53:56.178636+0000","last_clean":"2023-10-08T09:53:56.178636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T09:53:56.877269+0000","last_undegraded":"2023-10-08T09:53:56.877269+0000","last_fullsized":"2023-10-08T09:53:56.877269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242801+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:47.242801+0000","last_peered":"2023-10-08T09:54:47.242801+0000","last_clean":"2023-10-08T09:54:47.242801+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:47.242801+0000","last_undegraded":"2023-10-08T09:54:47.242801+0000","last_fullsized":"2023-10-08T09:54:47.242801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264616+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264616+0000","last_peered":"2023-10-08T09:54:47.264616+0000","last_clean":"2023-10-08T09:54:47.264616+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264616+0000","last_undegraded":"2023-10-08T09:54:47.264616+0000","last_fullsized":"2023-10-08T09:54:47.264616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235224+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:46.235224+0000","last_peered":"2023-10-08T09:54:46.235224+0000","last_clean":"2023-10-08T09:54:46.235224+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:46.235224+0000","last_undegraded":"2023-10-08T09:54:46.235224+0000","last_fullsized":"2023-10-08T09:54:46.235224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242196+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:47.242196+0000","last_peered":"2023-10-08T09:54:47.242196+0000","last_clean":"2023-10-08T09:54:47.242196+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:47.242196+0000","last_undegraded":"2023-10-08T09:54:47.242196+0000","last_fullsized":"2023-10-08T09:54:47.242196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235824+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:46.235824+0000","last_peered":"2023-10-08T09:54:46.235824+0000","last_clean":"2023-10-08T09:54:46.235824+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:46.235824+0000","last_undegraded":"2023-10-08T09:54:46.235824+0000","last_fullsized":"2023-10-08T09:54:46.235824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239283+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:46.239283+0000","last_peered":"2023-10-08T09:54:46.239283+0000","last_clean":"2023-10-08T09:54:46.239283+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:46.239283+0000","last_undegraded":"2023-10-08T09:54:46.239283+0000","last_fullsized":"2023-10-08T09:54:46.239283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236862+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:47.236862+0000","last_peered":"2023-10-08T09:54:47.236862+0000","last_clean":"2023-10-08T09:54:47.236862+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:47.236862+0000","last_undegraded":"2023-10-08T09:54:47.236862+0000","last_fullsized":"2023-10-08T09:54:47.236862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235382+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:46.235382+0000","last_peered":"2023-10-08T09:54:46.235382+0000","last_clean":"2023-10-08T09:54:46.235382+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:46.235382+0000","last_undegraded":"2023-10-08T09:54:46.235382+0000","last_fullsized":"2023-10-08T09:54:46.235382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242374+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:47.242374+0000","last_peered":"2023-10-08T09:54:47.242374+0000","last_clean":"2023-10-08T09:54:47.242374+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:47.242374+0000","last_undegraded":"2023-10-08T09:54:47.242374+0000","last_fullsized":"2023-10-08T09:54:47.242374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239156+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:46.239156+0000","last_peered":"2023-10-08T09:54:46.239156+0000","last_clean":"2023-10-08T09:54:46.239156+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:46.239156+0000","last_undegraded":"2023-10-08T09:54:46.239156+0000","last_fullsized":"2023-10-08T09:54:46.239156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235543+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:46.235543+0000","last_peered":"2023-10-08T09:54:46.235543+0000","last_clean":"2023-10-08T09:54:46.235543+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:46.235543+0000","last_undegraded":"2023-10-08T09:54:46.235543+0000","last_fullsized":"2023-10-08T09:54:46.235543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240249+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:46.240249+0000","last_peered":"2023-10-08T09:54:46.240249+0000","last_clean":"2023-10-08T09:54:46.240249+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:46.240249+0000","last_undegraded":"2023-10-08T09:54:46.240249+0000","last_fullsized":"2023-10-08T09:54:46.240249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236370+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:47.236370+0000","last_peered":"2023-10-08T09:54:47.236370+0000","last_clean":"2023-10-08T09:54:47.236370+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:47.236370+0000","last_undegraded":"2023-10-08T09:54:47.236370+0000","last_fullsized":"2023-10-08T09:54:47.236370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242860+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:47.242860+0000","last_peered":"2023-10-08T09:54:47.242860+0000","last_clean":"2023-10-08T09:54:47.242860+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:47.242860+0000","last_undegraded":"2023-10-08T09:54:47.242860+0000","last_fullsized":"2023-10-08T09:54:47.242860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239466+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:46.239466+0000","last_peered":"2023-10-08T09:54:46.239466+0000","last_clean":"2023-10-08T09:54:46.239466+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:46.239466+0000","last_undegraded":"2023-10-08T09:54:46.239466+0000","last_fullsized":"2023-10-08T09:54:46.239466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239080+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:46.239080+0000","last_peered":"2023-10-08T09:54:46.239080+0000","last_clean":"2023-10-08T09:54:46.239080+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:46.239080+0000","last_undegraded":"2023-10-08T09:54:46.239080+0000","last_fullsized":"2023-10-08T09:54:46.239080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236564+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:47.236564+0000","last_peered":"2023-10-08T09:54:47.236564+0000","last_clean":"2023-10-08T09:54:47.236564+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:47.236564+0000","last_undegraded":"2023-10-08T09:54:47.236564+0000","last_fullsized":"2023-10-08T09:54:47.236564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243492+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:47.243492+0000","last_peered":"2023-10-08T09:54:47.243492+0000","last_clean":"2023-10-08T09:54:47.243492+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:47.243492+0000","last_undegraded":"2023-10-08T09:54:47.243492+0000","last_fullsized":"2023-10-08T09:54:47.243492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239967+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:46.239967+0000","last_peered":"2023-10-08T09:54:46.239967+0000","last_clean":"2023-10-08T09:54:46.239967+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:46.239967+0000","last_undegraded":"2023-10-08T09:54:46.239967+0000","last_fullsized":"2023-10-08T09:54:46.239967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235937+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:46.235937+0000","last_peered":"2023-10-08T09:54:46.235937+0000","last_clean":"2023-10-08T09:54:46.235937+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:46.235937+0000","last_undegraded":"2023-10-08T09:54:46.235937+0000","last_fullsized":"2023-10-08T09:54:46.235937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235296+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:46.235296+0000","last_peered":"2023-10-08T09:54:46.235296+0000","last_clean":"2023-10-08T09:54:46.235296+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:46.235296+0000","last_undegraded":"2023-10-08T09:54:46.235296+0000","last_fullsized":"2023-10-08T09:54:46.235296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242750+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:47.242750+0000","last_peered":"2023-10-08T09:54:47.242750+0000","last_clean":"2023-10-08T09:54:47.242750+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:47.242750+0000","last_undegraded":"2023-10-08T09:54:47.242750+0000","last_fullsized":"2023-10-08T09:54:47.242750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":265,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":485589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.73899999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":0.879},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.85099999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.755},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.80400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1071443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.96999999999999997}]}]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.623},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.746}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81799999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.84699999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68300000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.745}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.81699999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59399999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85499999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.82399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.90500000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.623},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73099999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":489177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67600000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.878}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80500000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.95999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.315}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35199999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.44900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46700000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.60899999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":476633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.494},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":1.4299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.71099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.478},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":1.51}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63200000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.446},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":1.554}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.876}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":1.5349999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.5800000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":475679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.316}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36799999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54900000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55800000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.61299999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45700000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55300000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.78500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.71399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77100000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.48799999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:49.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:49 smithi186 ceph-mon[104393]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-08T09:55:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:49 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2295553847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:49 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2478590539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:49 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1178966003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 -- 172.21.15.97:0/4157530256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 msgr2=0x7fba400ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 --2- 172.21.15.97:0/4157530256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba400ff9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fba340099c0 tx=0x7fba34009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 -- 172.21.15.97:0/4157530256 shutdown_connections 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 --2- 172.21.15.97:0/4157530256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40101130 0x7fba401079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 --2- 172.21.15.97:0/4157530256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba400ff9c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 --2- 172.21.15.97:0/4157530256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40100790 0x7fba40100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 -- 172.21.15.97:0/4157530256 >> 172.21.15.97:0/4157530256 conn(0x7fba400fab10 msgr2=0x7fba400fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 -- 172.21.15.97:0/4157530256 shutdown_connections 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.654+0000 7fba448fc700 1 -- 172.21.15.97:0/4157530256 wait complete. 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 Processor -- start 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 -- start start 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40100790 0x7fba40101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40101130 0x7fba40102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40102a10 con 0x7fba400ff5a0 2023-10-08T09:55:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40102b80 con 0x7fba40101130 2023-10-08T09:55:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba448fc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba401a0650 con 0x7fba40100790 2023-10-08T09:55:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba3e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba3e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56984/0 (socket says 172.21.15.97:56984) 2023-10-08T09:55:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba3e59c700 1 -- 172.21.15.97:0/1139871061 learned_addr learned my addr 172.21.15.97:0/1139871061 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba3dd9b700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40100790 0x7fba40101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.655+0000 7fba3ed9d700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40101130 0x7fba40102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 -- 172.21.15.97:0/1139871061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40100790 msgr2=0x7fba40101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40100790 0x7fba40101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 -- 172.21.15.97:0/1139871061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40101130 msgr2=0x7fba40102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40101130 0x7fba40102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 -- 172.21.15.97:0/1139871061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba340095f0 con 0x7fba400ff5a0 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba3e59c700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fba34003820 tx=0x7fba34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3401d070 con 0x7fba400ff5a0 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba401a0880 con 0x7fba400ff5a0 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba340054f0 con 0x7fba400ff5a0 2023-10-08T09:55:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34004a00 con 0x7fba400ff5a0 2023-10-08T09:55:49.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.656+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba401a0cd0 con 0x7fba400ff5a0 2023-10-08T09:55:49.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.657+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba40067450 con 0x7fba400ff5a0 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.658+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fba34003b50 con 0x7fba400ff5a0 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.659+0000 7fba2f7fe700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 0x7fba280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.659+0000 7fba3dd9b700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 0x7fba280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.659+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba340ce280 con 0x7fba400ff5a0 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.659+0000 7fba3dd9b700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 0x7fba280a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba30005950 tx=0x7fba3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.663+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fba34097640 con 0x7fba400ff5a0 2023-10-08T09:55:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:49 smithi178 ceph-mon[103156]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-08T09:55:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:49 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2295553847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:49 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2478590539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:49 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1178966003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:49.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.850+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba40067f40 con 0x7fba280a1480 2023-10-08T09:55:49.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.862+0000 7fba2f7fe700 1 -- 172.21.15.97:0/1139871061 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427408 (secure 0 0 0) 0x7fba40067f40 con 0x7fba280a1480 2023-10-08T09:55:49.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 msgr2=0x7fba280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 0x7fba280a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba30005950 tx=0x7fba3000b410 comp rx=0 tx=0).stop 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 msgr2=0x7fba401036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fba34003820 tx=0x7fba34004730 comp rx=0 tx=0).stop 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 shutdown_connections 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fba280a1480 0x7fba280a3940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba40101130 0x7fba40102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba400ff5a0 0x7fba401036a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 --2- 172.21.15.97:0/1139871061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba40100790 0x7fba40101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 >> 172.21.15.97:0/1139871061 conn(0x7fba400fab10 msgr2=0x7fba40067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 shutdown_connections 2023-10-08T09:55:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:49.866+0000 7fba448fc700 1 -- 172.21.15.97:0/1139871061 wait complete. 2023-10-08T09:55:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:49.881 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2023-10-08T09:55:47.907130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":629,"num_read_kb":374,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337604,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604220,"statfs":{"total":1151940427776,"available":1151594721280,"internally_reserved":0,"allocated":13676544,"data_stored":7527424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002293"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243442+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:47.243442+0000","last_peered":"2023-10-08T09:54:47.243442+0000","last_clean":"2023-10-08T09:54:47.243442+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:47.243442+0000","last_undegraded":"2023-10-08T09:54:47.243442+0000","last_fullsized":"2023-10-08T09:54:47.243442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236793+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:47.236793+0000","last_peered":"2023-10-08T09:54:47.236793+0000","last_clean":"2023-10-08T09:54:47.236793+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:47.236793+0000","last_undegraded":"2023-10-08T09:54:47.236793+0000","last_fullsized":"2023-10-08T09:54:47.236793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.237042+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:47.237042+0000","last_peered":"2023-10-08T09:54:47.237042+0000","last_clean":"2023-10-08T09:54:47.237042+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:47.237042+0000","last_undegraded":"2023-10-08T09:54:47.237042+0000","last_fullsized":"2023-10-08T09:54:47.237042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235313+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:54:46.235313+0000","last_peered":"2023-10-08T09:54:46.235313+0000","last_clean":"2023-10-08T09:54:46.235313+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:54:46.235313+0000","last_undegraded":"2023-10-08T09:54:46.235313+0000","last_fullsized":"2023-10-08T09:54:46.235313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236665+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:47.236665+0000","last_peered":"2023-10-08T09:54:47.236665+0000","last_clean":"2023-10-08T09:54:47.236665+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:47.236665+0000","last_undegraded":"2023-10-08T09:54:47.236665+0000","last_fullsized":"2023-10-08T09:54:47.236665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236962+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:47.236962+0000","last_peered":"2023-10-08T09:54:47.236962+0000","last_clean":"2023-10-08T09:54:47.236962+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:47.236962+0000","last_undegraded":"2023-10-08T09:54:47.236962+0000","last_fullsized":"2023-10-08T09:54:47.236962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243351+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:47.243351+0000","last_peered":"2023-10-08T09:54:47.243351+0000","last_clean":"2023-10-08T09:54:47.243351+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:47.243351+0000","last_undegraded":"2023-10-08T09:54:47.243351+0000","last_fullsized":"2023-10-08T09:54:47.243351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242285+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:47.242285+0000","last_peered":"2023-10-08T09:54:47.242285+0000","last_clean":"2023-10-08T09:54:47.242285+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:47.242285+0000","last_undegraded":"2023-10-08T09:54:47.242285+0000","last_fullsized":"2023-10-08T09:54:47.242285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239781+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:54:46.239781+0000","last_peered":"2023-10-08T09:54:46.239781+0000","last_clean":"2023-10-08T09:54:46.239781+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:54:46.239781+0000","last_undegraded":"2023-10-08T09:54:46.239781+0000","last_fullsized":"2023-10-08T09:54:46.239781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372615+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:54:47.372615+0000","last_peered":"2023-10-08T09:54:47.372615+0000","last_clean":"2023-10-08T09:54:47.372615+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:54:47.372615+0000","last_undegraded":"2023-10-08T09:54:47.372615+0000","last_fullsized":"2023-10-08T09:54:47.372615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242460+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:47.242460+0000","last_peered":"2023-10-08T09:54:47.242460+0000","last_clean":"2023-10-08T09:54:47.242460+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:47.242460+0000","last_undegraded":"2023-10-08T09:54:47.242460+0000","last_fullsized":"2023-10-08T09:54:47.242460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235877+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:54:46.235877+0000","last_peered":"2023-10-08T09:54:46.235877+0000","last_clean":"2023-10-08T09:54:46.235877+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:54:46.235877+0000","last_undegraded":"2023-10-08T09:54:46.235877+0000","last_fullsized":"2023-10-08T09:54:46.235877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235918+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:54:46.235918+0000","last_peered":"2023-10-08T09:54:46.235918+0000","last_clean":"2023-10-08T09:54:46.235918+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:54:46.235918+0000","last_undegraded":"2023-10-08T09:54:46.235918+0000","last_fullsized":"2023-10-08T09:54:46.235918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236960+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:47.236960+0000","last_peered":"2023-10-08T09:54:47.236960+0000","last_clean":"2023-10-08T09:54:47.236960+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:47.236960+0000","last_undegraded":"2023-10-08T09:54:47.236960+0000","last_fullsized":"2023-10-08T09:54:47.236960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239642+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:54:46.239642+0000","last_peered":"2023-10-08T09:54:46.239642+0000","last_clean":"2023-10-08T09:54:46.239642+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:54:46.239642+0000","last_undegraded":"2023-10-08T09:54:46.239642+0000","last_fullsized":"2023-10-08T09:54:46.239642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.310191+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:43.310191+0000","last_peered":"2023-10-08T09:55:43.310191+0000","last_clean":"2023-10-08T09:55:43.310191+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:43.310191+0000","last_undegraded":"2023-10-08T09:55:43.310191+0000","last_fullsized":"2023-10-08T09:55:43.310191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239716+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:54:46.239716+0000","last_peered":"2023-10-08T09:54:46.239716+0000","last_clean":"2023-10-08T09:54:46.239716+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:54:46.239716+0000","last_undegraded":"2023-10-08T09:54:46.239716+0000","last_fullsized":"2023-10-08T09:54:46.239716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243249+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:47.243249+0000","last_peered":"2023-10-08T09:54:47.243249+0000","last_clean":"2023-10-08T09:54:47.243249+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:47.243249+0000","last_undegraded":"2023-10-08T09:54:47.243249+0000","last_fullsized":"2023-10-08T09:54:47.243249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235223+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:54:46.235223+0000","last_peered":"2023-10-08T09:54:46.235223+0000","last_clean":"2023-10-08T09:54:46.235223+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:54:46.235223+0000","last_undegraded":"2023-10-08T09:54:46.235223+0000","last_fullsized":"2023-10-08T09:54:46.235223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236698+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:47.236698+0000","last_peered":"2023-10-08T09:54:47.236698+0000","last_clean":"2023-10-08T09:54:47.236698+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:47.236698+0000","last_undegraded":"2023-10-08T09:54:47.236698+0000","last_fullsized":"2023-10-08T09:54:47.236698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242154+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:47.242154+0000","last_peered":"2023-10-08T09:54:47.242154+0000","last_clean":"2023-10-08T09:54:47.242154+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:47.242154+0000","last_undegraded":"2023-10-08T09:54:47.242154+0000","last_fullsized":"2023-10-08T09:54:47.242154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235537+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:54:46.235537+0000","last_peered":"2023-10-08T09:54:46.235537+0000","last_clean":"2023-10-08T09:54:46.235537+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:54:46.235537+0000","last_undegraded":"2023-10-08T09:54:46.235537+0000","last_fullsized":"2023-10-08T09:54:46.235537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.311163+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:44.311163+0000","last_peered":"2023-10-08T09:55:44.311163+0000","last_clean":"2023-10-08T09:55:44.311163+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:44.311163+0000","last_undegraded":"2023-10-08T09:55:44.311163+0000","last_fullsized":"2023-10-08T09:55:44.311163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.241714+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:47.241714+0000","last_peered":"2023-10-08T09:54:47.241714+0000","last_clean":"2023-10-08T09:54:47.241714+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:47.241714+0000","last_undegraded":"2023-10-08T09:54:47.241714+0000","last_fullsized":"2023-10-08T09:54:47.241714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236932+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:47.236932+0000","last_peered":"2023-10-08T09:54:47.236932+0000","last_clean":"2023-10-08T09:54:47.236932+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:47.236932+0000","last_undegraded":"2023-10-08T09:54:47.236932+0000","last_fullsized":"2023-10-08T09:54:47.236932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242039+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:47.242039+0000","last_peered":"2023-10-08T09:54:47.242039+0000","last_clean":"2023-10-08T09:54:47.242039+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:47.242039+0000","last_undegraded":"2023-10-08T09:54:47.242039+0000","last_fullsized":"2023-10-08T09:54:47.242039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":220,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:45.330927+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:45.330927+0000","last_peered":"2023-10-08T09:55:45.330927+0000","last_clean":"2023-10-08T09:55:45.330927+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:45.330927+0000","last_undegraded":"2023-10-08T09:55:45.330927+0000","last_fullsized":"2023-10-08T09:55:45.330927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":85,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240097+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:54:46.240097+0000","last_peered":"2023-10-08T09:54:46.240097+0000","last_clean":"2023-10-08T09:54:46.240097+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:54:46.240097+0000","last_undegraded":"2023-10-08T09:54:46.240097+0000","last_fullsized":"2023-10-08T09:54:46.240097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242355+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:47.242355+0000","last_peered":"2023-10-08T09:54:47.242355+0000","last_clean":"2023-10-08T09:54:47.242355+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:47.242355+0000","last_undegraded":"2023-10-08T09:54:47.242355+0000","last_fullsized":"2023-10-08T09:54:47.242355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242227+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:47.242227+0000","last_peered":"2023-10-08T09:54:47.242227+0000","last_clean":"2023-10-08T09:54:47.242227+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:47.242227+0000","last_undegraded":"2023-10-08T09:54:47.242227+0000","last_fullsized":"2023-10-08T09:54:47.242227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.309721+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:43.309721+0000","last_peered":"2023-10-08T09:55:43.309721+0000","last_clean":"2023-10-08T09:55:43.309721+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:43.309721+0000","last_undegraded":"2023-10-08T09:55:43.309721+0000","last_fullsized":"2023-10-08T09:55:43.309721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.268332+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.268332+0000","last_peered":"2023-10-08T09:54:47.268332+0000","last_clean":"2023-10-08T09:54:47.268332+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.268332+0000","last_undegraded":"2023-10-08T09:54:47.268332+0000","last_fullsized":"2023-10-08T09:54:47.268332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242632+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:47.242632+0000","last_peered":"2023-10-08T09:54:47.242632+0000","last_clean":"2023-10-08T09:54:47.242632+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:47.242632+0000","last_undegraded":"2023-10-08T09:54:47.242632+0000","last_fullsized":"2023-10-08T09:54:47.242632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242315+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:47.242315+0000","last_peered":"2023-10-08T09:54:47.242315+0000","last_clean":"2023-10-08T09:54:47.242315+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:47.242315+0000","last_undegraded":"2023-10-08T09:54:47.242315+0000","last_fullsized":"2023-10-08T09:54:47.242315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.310385+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:44.310385+0000","last_peered":"2023-10-08T09:55:44.310385+0000","last_clean":"2023-10-08T09:55:44.310385+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:44.310385+0000","last_undegraded":"2023-10-08T09:55:44.310385+0000","last_fullsized":"2023-10-08T09:55:44.310385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-10-08T09:53:56.877269+0000","last_change":"2023-10-08T09:55:46.965654+0000","last_active":"2023-10-08T09:53:56.178636+0000","last_peered":"2023-10-08T09:53:56.178636+0000","last_clean":"2023-10-08T09:53:56.178636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T09:53:56.877269+0000","last_undegraded":"2023-10-08T09:53:56.877269+0000","last_fullsized":"2023-10-08T09:53:56.877269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242801+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:47.242801+0000","last_peered":"2023-10-08T09:54:47.242801+0000","last_clean":"2023-10-08T09:54:47.242801+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:47.242801+0000","last_undegraded":"2023-10-08T09:54:47.242801+0000","last_fullsized":"2023-10-08T09:54:47.242801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264616+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264616+0000","last_peered":"2023-10-08T09:54:47.264616+0000","last_clean":"2023-10-08T09:54:47.264616+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264616+0000","last_undegraded":"2023-10-08T09:54:47.264616+0000","last_fullsized":"2023-10-08T09:54:47.264616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235224+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:54:46.235224+0000","last_peered":"2023-10-08T09:54:46.235224+0000","last_clean":"2023-10-08T09:54:46.235224+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:54:46.235224+0000","last_undegraded":"2023-10-08T09:54:46.235224+0000","last_fullsized":"2023-10-08T09:54:46.235224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242196+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:47.242196+0000","last_peered":"2023-10-08T09:54:47.242196+0000","last_clean":"2023-10-08T09:54:47.242196+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:47.242196+0000","last_undegraded":"2023-10-08T09:54:47.242196+0000","last_fullsized":"2023-10-08T09:54:47.242196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235824+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:54:46.235824+0000","last_peered":"2023-10-08T09:54:46.235824+0000","last_clean":"2023-10-08T09:54:46.235824+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:54:46.235824+0000","last_undegraded":"2023-10-08T09:54:46.235824+0000","last_fullsized":"2023-10-08T09:54:46.235824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239283+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:54:46.239283+0000","last_peered":"2023-10-08T09:54:46.239283+0000","last_clean":"2023-10-08T09:54:46.239283+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:54:46.239283+0000","last_undegraded":"2023-10-08T09:54:46.239283+0000","last_fullsized":"2023-10-08T09:54:46.239283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236862+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:47.236862+0000","last_peered":"2023-10-08T09:54:47.236862+0000","last_clean":"2023-10-08T09:54:47.236862+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:47.236862+0000","last_undegraded":"2023-10-08T09:54:47.236862+0000","last_fullsized":"2023-10-08T09:54:47.236862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235382+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:54:46.235382+0000","last_peered":"2023-10-08T09:54:46.235382+0000","last_clean":"2023-10-08T09:54:46.235382+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:54:46.235382+0000","last_undegraded":"2023-10-08T09:54:46.235382+0000","last_fullsized":"2023-10-08T09:54:46.235382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242374+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:47.242374+0000","last_peered":"2023-10-08T09:54:47.242374+0000","last_clean":"2023-10-08T09:54:47.242374+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:47.242374+0000","last_undegraded":"2023-10-08T09:54:47.242374+0000","last_fullsized":"2023-10-08T09:54:47.242374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239156+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:54:46.239156+0000","last_peered":"2023-10-08T09:54:46.239156+0000","last_clean":"2023-10-08T09:54:46.239156+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:54:46.239156+0000","last_undegraded":"2023-10-08T09:54:46.239156+0000","last_fullsized":"2023-10-08T09:54:46.239156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235543+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:54:46.235543+0000","last_peered":"2023-10-08T09:54:46.235543+0000","last_clean":"2023-10-08T09:54:46.235543+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:54:46.235543+0000","last_undegraded":"2023-10-08T09:54:46.235543+0000","last_fullsized":"2023-10-08T09:54:46.235543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240249+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:54:46.240249+0000","last_peered":"2023-10-08T09:54:46.240249+0000","last_clean":"2023-10-08T09:54:46.240249+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:54:46.240249+0000","last_undegraded":"2023-10-08T09:54:46.240249+0000","last_fullsized":"2023-10-08T09:54:46.240249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236370+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:47.236370+0000","last_peered":"2023-10-08T09:54:47.236370+0000","last_clean":"2023-10-08T09:54:47.236370+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:47.236370+0000","last_undegraded":"2023-10-08T09:54:47.236370+0000","last_fullsized":"2023-10-08T09:54:47.236370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242860+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:47.242860+0000","last_peered":"2023-10-08T09:54:47.242860+0000","last_clean":"2023-10-08T09:54:47.242860+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:47.242860+0000","last_undegraded":"2023-10-08T09:54:47.242860+0000","last_fullsized":"2023-10-08T09:54:47.242860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239466+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:54:46.239466+0000","last_peered":"2023-10-08T09:54:46.239466+0000","last_clean":"2023-10-08T09:54:46.239466+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:54:46.239466+0000","last_undegraded":"2023-10-08T09:54:46.239466+0000","last_fullsized":"2023-10-08T09:54:46.239466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239080+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:54:46.239080+0000","last_peered":"2023-10-08T09:54:46.239080+0000","last_clean":"2023-10-08T09:54:46.239080+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:54:46.239080+0000","last_undegraded":"2023-10-08T09:54:46.239080+0000","last_fullsized":"2023-10-08T09:54:46.239080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236564+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:47.236564+0000","last_peered":"2023-10-08T09:54:47.236564+0000","last_clean":"2023-10-08T09:54:47.236564+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:47.236564+0000","last_undegraded":"2023-10-08T09:54:47.236564+0000","last_fullsized":"2023-10-08T09:54:47.236564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243492+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:47.243492+0000","last_peered":"2023-10-08T09:54:47.243492+0000","last_clean":"2023-10-08T09:54:47.243492+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:47.243492+0000","last_undegraded":"2023-10-08T09:54:47.243492+0000","last_fullsized":"2023-10-08T09:54:47.243492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239967+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:54:46.239967+0000","last_peered":"2023-10-08T09:54:46.239967+0000","last_clean":"2023-10-08T09:54:46.239967+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:54:46.239967+0000","last_undegraded":"2023-10-08T09:54:46.239967+0000","last_fullsized":"2023-10-08T09:54:46.239967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235937+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:54:46.235937+0000","last_peered":"2023-10-08T09:54:46.235937+0000","last_clean":"2023-10-08T09:54:46.235937+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:54:46.235937+0000","last_undegraded":"2023-10-08T09:54:46.235937+0000","last_fullsized":"2023-10-08T09:54:46.235937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235296+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:54:46.235296+0000","last_peered":"2023-10-08T09:54:46.235296+0000","last_clean":"2023-10-08T09:54:46.235296+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:54:46.235296+0000","last_undegraded":"2023-10-08T09:54:46.235296+0000","last_fullsized":"2023-10-08T09:54:46.235296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242750+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:47.242750+0000","last_peered":"2023-10-08T09:54:47.242750+0000","last_clean":"2023-10-08T09:54:47.242750+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:47.242750+0000","last_undegraded":"2023-10-08T09:54:47.242750+0000","last_fullsized":"2023-10-08T09:54:47.242750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":265,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":485589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.73899999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":0.879},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.85099999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.755},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.80400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1071443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.96999999999999997}]}]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.623},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.746}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81799999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.84699999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68300000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.745}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.81699999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59399999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85499999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.82399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.90500000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.623},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73099999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":489177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67600000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.878}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80500000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.95999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.315}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35199999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.44900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46700000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.60899999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":476633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.494},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":1.4299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.71099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.478},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":1.51}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63200000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.446},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":1.554}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.876}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":1.5349999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.5800000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.92100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":475679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.316}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36799999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71499999999999997,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54900000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55800000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.61299999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45700000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55300000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.78500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.71399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77100000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.48799999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:49.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1473687526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ff5a0 msgr2=0x7fbcf40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1473687526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf40ff9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009a40 tx=0x7fbce4009d50 comp rx=0 tx=0).stop 2023-10-08T09:55:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1473687526 shutdown_connections 2023-10-08T09:55:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1473687526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf4101130 0x7fbcf41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1473687526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf40ff9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1473687526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 0x7fbcf4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1473687526 >> 172.21.15.97:0/1473687526 conn(0x7fbcf40fab10 msgr2=0x7fbcf40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1473687526 shutdown_connections 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1473687526 wait complete. 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.048+0000 7fbcf9fc9700 1 Processor -- start 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 -- start start 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf4194380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 0x7fbcf41948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419beb0 con 0x7fbcf4101130 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419bff0 con 0x7fbcf40ff5a0 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf9fc9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419c180 con 0x7fbcf4100790 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57002/0 (socket says 172.21.15.97:57002) 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 -- 172.21.15.97:0/1802148690 learned_addr learned my addr 172.21.15.97:0/1802148690 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf2ffd700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 0x7fbcf41948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf37fe700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf4194380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 -- 172.21.15.97:0/1802148690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 msgr2=0x7fbcf41948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 0x7fbcf41948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 -- 172.21.15.97:0/1802148690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 msgr2=0x7fbcf4194380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf4194380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 -- 172.21.15.97:0/1802148690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40095f0 con 0x7fbcf4101130 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf37fe700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf4194380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf3fff700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbcec007d70 tx=0x7fbcec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.049+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00ad40 con 0x7fbcf4101130 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.050+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf419c460 con 0x7fbcf4101130 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.050+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcec0128e0 con 0x7fbcf4101130 2023-10-08T09:55:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.050+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec011780 con 0x7fbcf4101130 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.050+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf419c970 con 0x7fbcf4101130 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.050+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf4106950 con 0x7fbcf4101130 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.051+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbcec012a50 con 0x7fbcf4101130 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.052+0000 7fbcf0ff9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 0x7fbce00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.052+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbcec0cd7e0 con 0x7fbcf4101130 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.052+0000 7fbcf37fe700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 0x7fbce00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.053+0000 7fbcf37fe700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 0x7fbce00a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009960 tx=0x7fbce4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.056+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbcec096b10 con 0x7fbcf4101130 2023-10-08T09:55:50.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.248+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcf4060c30 con 0x7fbce00a1480 2023-10-08T09:55:50.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.260+0000 7fbcf0ff9700 1 -- 172.21.15.97:0/1802148690 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427456 (secure 0 0 0) 0x7fbcf4060c30 con 0x7fbce00a1480 2023-10-08T09:55:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:50.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 msgr2=0x7fbce00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 0x7fbce00a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009960 tx=0x7fbce4010040 comp rx=0 tx=0).stop 2023-10-08T09:55:50.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 msgr2=0x7fbcf419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbcec007d70 tx=0x7fbcec00a340 comp rx=0 tx=0).stop 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 shutdown_connections 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbce00a1480 0x7fbce00a3940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf4101130 0x7fbcf419b8e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcf40ff5a0 0x7fbcf4194380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 --2- 172.21.15.97:0/1802148690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcf4100790 0x7fbcf41948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.264+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 >> 172.21.15.97:0/1802148690 conn(0x7fbcf40fab10 msgr2=0x7fbcf4103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.265+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 shutdown_connections 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.265+0000 7fbcf9fc9700 1 -- 172.21.15.97:0/1802148690 wait complete. 2023-10-08T09:55:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2023-10-08T09:55:49.907687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":629,"num_read_kb":374,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337672,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604152,"statfs":{"total":1151940427776,"available":1151594651648,"internally_reserved":0,"allocated":13742080,"data_stored":7565712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002363"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243442+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:47.243442+0000","last_peered":"2023-10-08T09:54:47.243442+0000","last_clean":"2023-10-08T09:54:47.243442+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:47.243442+0000","last_undegraded":"2023-10-08T09:54:47.243442+0000","last_fullsized":"2023-10-08T09:54:47.243442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236793+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:47.236793+0000","last_peered":"2023-10-08T09:54:47.236793+0000","last_clean":"2023-10-08T09:54:47.236793+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:47.236793+0000","last_undegraded":"2023-10-08T09:54:47.236793+0000","last_fullsized":"2023-10-08T09:54:47.236793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.237042+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:47.237042+0000","last_peered":"2023-10-08T09:54:47.237042+0000","last_clean":"2023-10-08T09:54:47.237042+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:47.237042+0000","last_undegraded":"2023-10-08T09:54:47.237042+0000","last_fullsized":"2023-10-08T09:54:47.237042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321405+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:55:46.321405+0000","last_peered":"2023-10-08T09:55:46.321405+0000","last_clean":"2023-10-08T09:55:46.321405+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:55:46.321405+0000","last_undegraded":"2023-10-08T09:55:46.321405+0000","last_fullsized":"2023-10-08T09:55:46.321405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236665+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:47.236665+0000","last_peered":"2023-10-08T09:54:47.236665+0000","last_clean":"2023-10-08T09:54:47.236665+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:47.236665+0000","last_undegraded":"2023-10-08T09:54:47.236665+0000","last_fullsized":"2023-10-08T09:54:47.236665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236962+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:47.236962+0000","last_peered":"2023-10-08T09:54:47.236962+0000","last_clean":"2023-10-08T09:54:47.236962+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:47.236962+0000","last_undegraded":"2023-10-08T09:54:47.236962+0000","last_fullsized":"2023-10-08T09:54:47.236962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243351+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:47.243351+0000","last_peered":"2023-10-08T09:54:47.243351+0000","last_clean":"2023-10-08T09:54:47.243351+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:47.243351+0000","last_undegraded":"2023-10-08T09:54:47.243351+0000","last_fullsized":"2023-10-08T09:54:47.243351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242285+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:47.242285+0000","last_peered":"2023-10-08T09:54:47.242285+0000","last_clean":"2023-10-08T09:54:47.242285+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:47.242285+0000","last_undegraded":"2023-10-08T09:54:47.242285+0000","last_fullsized":"2023-10-08T09:54:47.242285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787058+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:55:46.787058+0000","last_peered":"2023-10-08T09:55:46.787058+0000","last_clean":"2023-10-08T09:55:46.787058+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:55:46.787058+0000","last_undegraded":"2023-10-08T09:55:46.787058+0000","last_fullsized":"2023-10-08T09:55:46.787058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786778+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:55:46.786778+0000","last_peered":"2023-10-08T09:55:46.786778+0000","last_clean":"2023-10-08T09:55:46.786778+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:55:46.786778+0000","last_undegraded":"2023-10-08T09:55:46.786778+0000","last_fullsized":"2023-10-08T09:55:46.786778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242460+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:47.242460+0000","last_peered":"2023-10-08T09:54:47.242460+0000","last_clean":"2023-10-08T09:54:47.242460+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:47.242460+0000","last_undegraded":"2023-10-08T09:54:47.242460+0000","last_fullsized":"2023-10-08T09:54:47.242460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321977+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:55:46.321977+0000","last_peered":"2023-10-08T09:55:46.321977+0000","last_clean":"2023-10-08T09:55:46.321977+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:55:46.321977+0000","last_undegraded":"2023-10-08T09:55:46.321977+0000","last_fullsized":"2023-10-08T09:55:46.321977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322041+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:55:46.322041+0000","last_peered":"2023-10-08T09:55:46.322041+0000","last_clean":"2023-10-08T09:55:46.322041+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:55:46.322041+0000","last_undegraded":"2023-10-08T09:55:46.322041+0000","last_fullsized":"2023-10-08T09:55:46.322041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236960+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:47.236960+0000","last_peered":"2023-10-08T09:54:47.236960+0000","last_clean":"2023-10-08T09:54:47.236960+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:47.236960+0000","last_undegraded":"2023-10-08T09:54:47.236960+0000","last_fullsized":"2023-10-08T09:54:47.236960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786977+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:55:46.786977+0000","last_peered":"2023-10-08T09:55:46.786977+0000","last_clean":"2023-10-08T09:55:46.786977+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:55:46.786977+0000","last_undegraded":"2023-10-08T09:55:46.786977+0000","last_fullsized":"2023-10-08T09:55:46.786977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.310191+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:43.310191+0000","last_peered":"2023-10-08T09:55:43.310191+0000","last_clean":"2023-10-08T09:55:43.310191+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:43.310191+0000","last_undegraded":"2023-10-08T09:55:43.310191+0000","last_fullsized":"2023-10-08T09:55:43.310191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786805+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:55:46.786805+0000","last_peered":"2023-10-08T09:55:46.786805+0000","last_clean":"2023-10-08T09:55:46.786805+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:55:46.786805+0000","last_undegraded":"2023-10-08T09:55:46.786805+0000","last_fullsized":"2023-10-08T09:55:46.786805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243249+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:47.243249+0000","last_peered":"2023-10-08T09:54:47.243249+0000","last_clean":"2023-10-08T09:54:47.243249+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:47.243249+0000","last_undegraded":"2023-10-08T09:54:47.243249+0000","last_fullsized":"2023-10-08T09:54:47.243249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321353+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:55:46.321353+0000","last_peered":"2023-10-08T09:55:46.321353+0000","last_clean":"2023-10-08T09:55:46.321353+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:55:46.321353+0000","last_undegraded":"2023-10-08T09:55:46.321353+0000","last_fullsized":"2023-10-08T09:55:46.321353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236698+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:47.236698+0000","last_peered":"2023-10-08T09:54:47.236698+0000","last_clean":"2023-10-08T09:54:47.236698+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:47.236698+0000","last_undegraded":"2023-10-08T09:54:47.236698+0000","last_fullsized":"2023-10-08T09:54:47.236698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242154+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:47.242154+0000","last_peered":"2023-10-08T09:54:47.242154+0000","last_clean":"2023-10-08T09:54:47.242154+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:47.242154+0000","last_undegraded":"2023-10-08T09:54:47.242154+0000","last_fullsized":"2023-10-08T09:54:47.242154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321535+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:55:46.321535+0000","last_peered":"2023-10-08T09:55:46.321535+0000","last_clean":"2023-10-08T09:55:46.321535+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:55:46.321535+0000","last_undegraded":"2023-10-08T09:55:46.321535+0000","last_fullsized":"2023-10-08T09:55:46.321535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.311163+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:44.311163+0000","last_peered":"2023-10-08T09:55:44.311163+0000","last_clean":"2023-10-08T09:55:44.311163+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:44.311163+0000","last_undegraded":"2023-10-08T09:55:44.311163+0000","last_fullsized":"2023-10-08T09:55:44.311163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.241714+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:47.241714+0000","last_peered":"2023-10-08T09:54:47.241714+0000","last_clean":"2023-10-08T09:54:47.241714+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:47.241714+0000","last_undegraded":"2023-10-08T09:54:47.241714+0000","last_fullsized":"2023-10-08T09:54:47.241714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236932+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:47.236932+0000","last_peered":"2023-10-08T09:54:47.236932+0000","last_clean":"2023-10-08T09:54:47.236932+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:47.236932+0000","last_undegraded":"2023-10-08T09:54:47.236932+0000","last_fullsized":"2023-10-08T09:54:47.236932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242039+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:47.242039+0000","last_peered":"2023-10-08T09:54:47.242039+0000","last_clean":"2023-10-08T09:54:47.242039+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:47.242039+0000","last_undegraded":"2023-10-08T09:54:47.242039+0000","last_fullsized":"2023-10-08T09:54:47.242039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":220,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:45.330927+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:45.330927+0000","last_peered":"2023-10-08T09:55:45.330927+0000","last_clean":"2023-10-08T09:55:45.330927+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:45.330927+0000","last_undegraded":"2023-10-08T09:55:45.330927+0000","last_fullsized":"2023-10-08T09:55:45.330927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":85,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786794+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:55:46.786794+0000","last_peered":"2023-10-08T09:55:46.786794+0000","last_clean":"2023-10-08T09:55:46.786794+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:55:46.786794+0000","last_undegraded":"2023-10-08T09:55:46.786794+0000","last_fullsized":"2023-10-08T09:55:46.786794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242355+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:47.242355+0000","last_peered":"2023-10-08T09:54:47.242355+0000","last_clean":"2023-10-08T09:54:47.242355+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:47.242355+0000","last_undegraded":"2023-10-08T09:54:47.242355+0000","last_fullsized":"2023-10-08T09:54:47.242355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242227+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:47.242227+0000","last_peered":"2023-10-08T09:54:47.242227+0000","last_clean":"2023-10-08T09:54:47.242227+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:47.242227+0000","last_undegraded":"2023-10-08T09:54:47.242227+0000","last_fullsized":"2023-10-08T09:54:47.242227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.309721+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:43.309721+0000","last_peered":"2023-10-08T09:55:43.309721+0000","last_clean":"2023-10-08T09:55:43.309721+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:43.309721+0000","last_undegraded":"2023-10-08T09:55:43.309721+0000","last_fullsized":"2023-10-08T09:55:43.309721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.268332+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.268332+0000","last_peered":"2023-10-08T09:54:47.268332+0000","last_clean":"2023-10-08T09:54:47.268332+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.268332+0000","last_undegraded":"2023-10-08T09:54:47.268332+0000","last_fullsized":"2023-10-08T09:54:47.268332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242632+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:47.242632+0000","last_peered":"2023-10-08T09:54:47.242632+0000","last_clean":"2023-10-08T09:54:47.242632+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:47.242632+0000","last_undegraded":"2023-10-08T09:54:47.242632+0000","last_fullsized":"2023-10-08T09:54:47.242632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242315+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:47.242315+0000","last_peered":"2023-10-08T09:54:47.242315+0000","last_clean":"2023-10-08T09:54:47.242315+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:47.242315+0000","last_undegraded":"2023-10-08T09:54:47.242315+0000","last_fullsized":"2023-10-08T09:54:47.242315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.310385+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:44.310385+0000","last_peered":"2023-10-08T09:55:44.310385+0000","last_clean":"2023-10-08T09:55:44.310385+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:44.310385+0000","last_undegraded":"2023-10-08T09:55:44.310385+0000","last_fullsized":"2023-10-08T09:55:44.310385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-10-08T09:53:56.877269+0000","last_change":"2023-10-08T09:55:46.965654+0000","last_active":"2023-10-08T09:53:56.178636+0000","last_peered":"2023-10-08T09:53:56.178636+0000","last_clean":"2023-10-08T09:53:56.178636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T09:53:56.877269+0000","last_undegraded":"2023-10-08T09:53:56.877269+0000","last_fullsized":"2023-10-08T09:53:56.877269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242801+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:47.242801+0000","last_peered":"2023-10-08T09:54:47.242801+0000","last_clean":"2023-10-08T09:54:47.242801+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:47.242801+0000","last_undegraded":"2023-10-08T09:54:47.242801+0000","last_fullsized":"2023-10-08T09:54:47.242801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264616+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264616+0000","last_peered":"2023-10-08T09:54:47.264616+0000","last_clean":"2023-10-08T09:54:47.264616+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264616+0000","last_undegraded":"2023-10-08T09:54:47.264616+0000","last_fullsized":"2023-10-08T09:54:47.264616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321320+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:55:46.321320+0000","last_peered":"2023-10-08T09:55:46.321320+0000","last_clean":"2023-10-08T09:55:46.321320+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:55:46.321320+0000","last_undegraded":"2023-10-08T09:55:46.321320+0000","last_fullsized":"2023-10-08T09:55:46.321320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242196+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:47.242196+0000","last_peered":"2023-10-08T09:54:47.242196+0000","last_clean":"2023-10-08T09:54:47.242196+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:47.242196+0000","last_undegraded":"2023-10-08T09:54:47.242196+0000","last_fullsized":"2023-10-08T09:54:47.242196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321884+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:55:46.321884+0000","last_peered":"2023-10-08T09:55:46.321884+0000","last_clean":"2023-10-08T09:55:46.321884+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:55:46.321884+0000","last_undegraded":"2023-10-08T09:55:46.321884+0000","last_fullsized":"2023-10-08T09:55:46.321884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786506+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:55:46.786506+0000","last_peered":"2023-10-08T09:55:46.786506+0000","last_clean":"2023-10-08T09:55:46.786506+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:55:46.786506+0000","last_undegraded":"2023-10-08T09:55:46.786506+0000","last_fullsized":"2023-10-08T09:55:46.786506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236862+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:47.236862+0000","last_peered":"2023-10-08T09:54:47.236862+0000","last_clean":"2023-10-08T09:54:47.236862+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:47.236862+0000","last_undegraded":"2023-10-08T09:54:47.236862+0000","last_fullsized":"2023-10-08T09:54:47.236862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321433+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:55:46.321433+0000","last_peered":"2023-10-08T09:55:46.321433+0000","last_clean":"2023-10-08T09:55:46.321433+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:55:46.321433+0000","last_undegraded":"2023-10-08T09:55:46.321433+0000","last_fullsized":"2023-10-08T09:55:46.321433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242374+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:47.242374+0000","last_peered":"2023-10-08T09:54:47.242374+0000","last_clean":"2023-10-08T09:54:47.242374+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:47.242374+0000","last_undegraded":"2023-10-08T09:54:47.242374+0000","last_fullsized":"2023-10-08T09:54:47.242374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786643+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:55:46.786643+0000","last_peered":"2023-10-08T09:55:46.786643+0000","last_clean":"2023-10-08T09:55:46.786643+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:55:46.786643+0000","last_undegraded":"2023-10-08T09:55:46.786643+0000","last_fullsized":"2023-10-08T09:55:46.786643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321742+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:55:46.321742+0000","last_peered":"2023-10-08T09:55:46.321742+0000","last_clean":"2023-10-08T09:55:46.321742+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:55:46.321742+0000","last_undegraded":"2023-10-08T09:55:46.321742+0000","last_fullsized":"2023-10-08T09:55:46.321742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786917+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:55:46.786917+0000","last_peered":"2023-10-08T09:55:46.786917+0000","last_clean":"2023-10-08T09:55:46.786917+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:55:46.786917+0000","last_undegraded":"2023-10-08T09:55:46.786917+0000","last_fullsized":"2023-10-08T09:55:46.786917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236370+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:47.236370+0000","last_peered":"2023-10-08T09:54:47.236370+0000","last_clean":"2023-10-08T09:54:47.236370+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:47.236370+0000","last_undegraded":"2023-10-08T09:54:47.236370+0000","last_fullsized":"2023-10-08T09:54:47.236370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242860+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:47.242860+0000","last_peered":"2023-10-08T09:54:47.242860+0000","last_clean":"2023-10-08T09:54:47.242860+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:47.242860+0000","last_undegraded":"2023-10-08T09:54:47.242860+0000","last_fullsized":"2023-10-08T09:54:47.242860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786636+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:55:46.786636+0000","last_peered":"2023-10-08T09:55:46.786636+0000","last_clean":"2023-10-08T09:55:46.786636+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:55:46.786636+0000","last_undegraded":"2023-10-08T09:55:46.786636+0000","last_fullsized":"2023-10-08T09:55:46.786636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786645+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:55:46.786645+0000","last_peered":"2023-10-08T09:55:46.786645+0000","last_clean":"2023-10-08T09:55:46.786645+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:55:46.786645+0000","last_undegraded":"2023-10-08T09:55:46.786645+0000","last_fullsized":"2023-10-08T09:55:46.786645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236564+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:47.236564+0000","last_peered":"2023-10-08T09:54:47.236564+0000","last_clean":"2023-10-08T09:54:47.236564+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:47.236564+0000","last_undegraded":"2023-10-08T09:54:47.236564+0000","last_fullsized":"2023-10-08T09:54:47.236564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243492+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:47.243492+0000","last_peered":"2023-10-08T09:54:47.243492+0000","last_clean":"2023-10-08T09:54:47.243492+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:47.243492+0000","last_undegraded":"2023-10-08T09:54:47.243492+0000","last_fullsized":"2023-10-08T09:54:47.243492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787148+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:55:46.787148+0000","last_peered":"2023-10-08T09:55:46.787148+0000","last_clean":"2023-10-08T09:55:46.787148+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:55:46.787148+0000","last_undegraded":"2023-10-08T09:55:46.787148+0000","last_fullsized":"2023-10-08T09:55:46.787148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321983+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:55:46.321983+0000","last_peered":"2023-10-08T09:55:46.321983+0000","last_clean":"2023-10-08T09:55:46.321983+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:55:46.321983+0000","last_undegraded":"2023-10-08T09:55:46.321983+0000","last_fullsized":"2023-10-08T09:55:46.321983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321400+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:55:46.321400+0000","last_peered":"2023-10-08T09:55:46.321400+0000","last_clean":"2023-10-08T09:55:46.321400+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:55:46.321400+0000","last_undegraded":"2023-10-08T09:55:46.321400+0000","last_fullsized":"2023-10-08T09:55:46.321400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242750+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:47.242750+0000","last_peered":"2023-10-08T09:54:47.242750+0000","last_clean":"2023-10-08T09:54:47.242750+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:47.242750+0000","last_undegraded":"2023-10-08T09:54:47.242750+0000","last_fullsized":"2023-10-08T09:54:47.242750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":265,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":485589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.73899999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":0.879},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.85099999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.755},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.80400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1071443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.96999999999999997}]}]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.623},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.746}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81799999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.84699999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68300000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.745}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.81699999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59399999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85499999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.82399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.90500000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.623},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73099999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":489177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67600000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.878}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80500000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.95999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.315}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35199999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.44900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46700000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.60899999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":495777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.88500000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.94299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.92700000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.008}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":1.3959999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":1.919}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":4.1840000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":5.0140000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.99199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":5.8659999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5170000000000003,"5min":6.5170000000000003,"15min":6.5170000000000003},"last":6.5170000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.1279999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":7.2649999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":8.1259999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.014}]}]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45700000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55300000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.78500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.71399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77100000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.48799999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:50.291 INFO:tasks.fwd_scrub.ceph_manager:PG 3.2 is not active+clean 2023-10-08T09:55:50.292 INFO:tasks.fwd_scrub.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-10-08T09:53:56.877269+0000', 'last_change': '2023-10-08T09:55:46.965654+0000', 'last_active': '2023-10-08T09:53:56.178636+0000', 'last_peered': '2023-10-08T09:53:56.178636+0000', 'last_clean': '2023-10-08T09:53:56.178636+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T09:53:56.877269+0000', 'last_undegraded': '2023-10-08T09:53:56.877269+0000', 'last_fullsized': '2023-10-08T09:53:56.877269+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 80, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T09:53:56.178636+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T09:53:56.178636+0000', 'last_clean_scrub_stamp': '2023-10-08T09:53:56.178636+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 5, 1], 'acting': [9, 5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-10-08T09:55:50.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T09:55:50.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.437+0000 7f01a9284700 1 -- 172.21.15.97:0/2184292829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 msgr2=0x7f01a4100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.437+0000 7f01a9284700 1 --2- 172.21.15.97:0/2184292829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a4100c20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f01940099c0 tx=0x7f0194009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- 172.21.15.97:0/2184292829 shutdown_connections 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- 172.21.15.97:0/2184292829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a4101160 0x7f01a4107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- 172.21.15.97:0/2184292829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a40ff5d0 0x7f01a40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- 172.21.15.97:0/2184292829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a4100c20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- 172.21.15.97:0/2184292829 >> 172.21.15.97:0/2184292829 conn(0x7f01a40fab80 msgr2=0x7f01a40fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- 172.21.15.97:0/2184292829 shutdown_connections 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- 172.21.15.97:0/2184292829 wait complete. 2023-10-08T09:55:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 Processor -- start 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- start start 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 0x7f01a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a4101160 0x7f01a419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419e170 con 0x7f01a41007c0 2023-10-08T09:55:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419e2e0 con 0x7f01a4101160 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.438+0000 7f01a9284700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a419e4a0 con 0x7f01a40ff5d0 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57012/0 (socket says 172.21.15.97:57012) 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 -- 172.21.15.97:0/2724710928 learned_addr learned my addr 172.21.15.97:0/2724710928 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a2ffd700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 0x7f01a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 -- 172.21.15.97:0/2724710928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 msgr2=0x7f01a40697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 0x7f01a40697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a37fe700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a4101160 0x7f01a419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 -- 172.21.15.97:0/2724710928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a4101160 msgr2=0x7f01a419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a4101160 0x7f01a419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 -- 172.21.15.97:0/2724710928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940095f0 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a2ffd700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 0x7f01a40697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a27fc700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f01940036f0 tx=0x7f01940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019401d070 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0194004110 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a419e6c0 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194022e10 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.439+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a419eb70 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.440+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4106a20 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.441+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0194022470 con 0x7f01a41007c0 2023-10-08T09:55:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.442+0000 7f018bfff700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 0x7f018c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.442+0000 7f01a2ffd700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 0x7f018c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.442+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01940ce560 con 0x7f01a41007c0 2023-10-08T09:55:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.442+0000 7f01a2ffd700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 0x7f018c0a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f0198005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.446+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0194097920 con 0x7f01a41007c0 2023-10-08T09:55:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.721+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f01a40678a0 con 0x7f01a41007c0 2023-10-08T09:55:50.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.722+0000 7f018bfff700 1 -- 172.21.15.97:0/2724710928 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1721 (secure 0 0 0) 0x7f019409af40 con 0x7f01a41007c0 2023-10-08T09:55:50.724 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:50.724 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive, 1 pg peering","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":435,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1696758796,"num_in_osds":12,"osd_in_since":1696758779,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"peering","count":1}],"num_pgs":129,"num_pools":4,"num_objects":62,"data_bytes":598080,"bytes_used":345776128,"bytes_avail":1151594651648,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"read_bytes_sec":2047,"write_bytes_sec":255,"read_op_per_sec":3,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24518},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24532},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24536},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24541},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24562},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24566}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.97:8443/"}},"servicemap":{"epoch":27,"modified":"2023-10-08T09:55:31.903016+0000","services":{}},"progress_events":{}} 2023-10-08T09:55:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 msgr2=0x7f018c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 0x7f018c0a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f0198005f00 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 msgr2=0x7f01a419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f01940036f0 tx=0x7f01940047b0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 shutdown_connections 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f018c0a1480 0x7f018c0a3940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a4101160 0x7f01a419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a40ff5d0 0x7f01a40697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 --2- 172.21.15.97:0/2724710928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01a41007c0 0x7f01a419b360 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 >> 172.21.15.97:0/2724710928 conn(0x7f01a40fab80 msgr2=0x7f01a4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 shutdown_connections 2023-10-08T09:55:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.724+0000 7f01a9284700 1 -- 172.21.15.97:0/2724710928 wait complete. 2023-10-08T09:55:50.738 INFO:tasks.fwd_scrub.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T09:55:50.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 -- 172.21.15.97:0/525194888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100b60 msgr2=0x7f386c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 --2- 172.21.15.97:0/525194888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100b60 0x7f386c100f80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3860009990 tx=0x7f3860009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 -- 172.21.15.97:0/525194888 shutdown_connections 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 --2- 172.21.15.97:0/525194888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100170 0x7f386c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 --2- 172.21.15.97:0/525194888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100b60 0x7f386c100f80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 --2- 172.21.15.97:0/525194888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 0x7f386c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 -- 172.21.15.97:0/525194888 >> 172.21.15.97:0/525194888 conn(0x7f386c0fa2c0 msgr2=0x7f386c0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 -- 172.21.15.97:0/525194888 shutdown_connections 2023-10-08T09:55:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 -- 172.21.15.97:0/525194888 wait complete. 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.858+0000 7f38717f7700 1 Processor -- start 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 -- start start 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 0x7f386c1954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100b60 0x7f386c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c19cff0 con 0x7f386c100170 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c19d130 con 0x7f386c0ff7b0 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c19d2c0 con 0x7f386c100b60 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57026/0 (socket says 172.21.15.97:57026) 2023-10-08T09:55:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 -- 172.21.15.97:0/3735666364 learned_addr learned my addr 172.21.15.97:0/3735666364 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386affd700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 0x7f386c1954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386b7fe700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100b60 0x7f386c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 -- 172.21.15.97:0/3735666364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100b60 msgr2=0x7f386c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100b60 0x7f386c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 -- 172.21.15.97:0/3735666364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 msgr2=0x7f386c1954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 0x7f386c1954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 -- 172.21.15.97:0/3735666364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600095f0 con 0x7f386c100170 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f386a7fc700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f385c00d950 tx=0x7f385c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c009b40 con 0x7f386c100170 2023-10-08T09:55:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c19d5a0 con 0x7f386c100170 2023-10-08T09:55:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385c009ca0 con 0x7f386c100170 2023-10-08T09:55:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.859+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00f620 con 0x7f386c100170 2023-10-08T09:55:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.860+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c19db00 con 0x7f386c100170 2023-10-08T09:55:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.860+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c106a20 con 0x7f386c100170 2023-10-08T09:55:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.861+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f385c010460 con 0x7f386c100170 2023-10-08T09:55:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.862+0000 7f3853fff700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 0x7f38540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.862+0000 7f386affd700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 0x7f38540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.862+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f385c0cd320 con 0x7f386c100170 2023-10-08T09:55:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.862+0000 7f386affd700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 0x7f38540a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38600037f0 tx=0x7f386000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:50.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:50.866+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f385c0966f0 con 0x7f386c100170 2023-10-08T09:55:51.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.054+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f386c060c30 con 0x7f38540a1480 2023-10-08T09:55:51.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.065+0000 7f3853fff700 1 -- 172.21.15.97:0/3735666364 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427456 (secure 0 0 0) 0x7f386c060c30 con 0x7f38540a1480 2023-10-08T09:55:51.073 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 msgr2=0x7f38540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 0x7f38540a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38600037f0 tx=0x7f386000b540 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 msgr2=0x7f386c195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f385c00d950 tx=0x7f385c00dd10 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 shutdown_connections 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f38540a1480 0x7f38540a3940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f386c100b60 0x7f386c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f386c0ff7b0 0x7f386c1954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 --2- 172.21.15.97:0/3735666364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c100170 0x7f386c195a00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.069+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 >> 172.21.15.97:0/3735666364 conn(0x7f386c0fa2c0 msgr2=0x7f386c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.070+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 shutdown_connections 2023-10-08T09:55:51.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:51.070+0000 7f38717f7700 1 -- 172.21.15.97:0/3735666364 wait complete. 2023-10-08T09:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:51.084 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2023-10-08T09:55:49.907687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":629,"num_read_kb":374,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337672,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604152,"statfs":{"total":1151940427776,"available":1151594651648,"internally_reserved":0,"allocated":13742080,"data_stored":7565712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002363"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243442+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:54:47.243442+0000","last_peered":"2023-10-08T09:54:47.243442+0000","last_clean":"2023-10-08T09:54:47.243442+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:54:47.243442+0000","last_undegraded":"2023-10-08T09:54:47.243442+0000","last_fullsized":"2023-10-08T09:54:47.243442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236793+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:54:47.236793+0000","last_peered":"2023-10-08T09:54:47.236793+0000","last_clean":"2023-10-08T09:54:47.236793+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:54:47.236793+0000","last_undegraded":"2023-10-08T09:54:47.236793+0000","last_fullsized":"2023-10-08T09:54:47.236793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.237042+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:54:47.237042+0000","last_peered":"2023-10-08T09:54:47.237042+0000","last_clean":"2023-10-08T09:54:47.237042+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:54:47.237042+0000","last_undegraded":"2023-10-08T09:54:47.237042+0000","last_fullsized":"2023-10-08T09:54:47.237042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240691+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:54:46.240691+0000","last_peered":"2023-10-08T09:54:46.240691+0000","last_clean":"2023-10-08T09:54:46.240691+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:54:46.240691+0000","last_undegraded":"2023-10-08T09:54:46.240691+0000","last_fullsized":"2023-10-08T09:54:46.240691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235428+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:54:46.235428+0000","last_peered":"2023-10-08T09:54:46.235428+0000","last_clean":"2023-10-08T09:54:46.235428+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:54:46.235428+0000","last_undegraded":"2023-10-08T09:54:46.235428+0000","last_fullsized":"2023-10-08T09:54:46.235428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239025+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:54:46.239025+0000","last_peered":"2023-10-08T09:54:46.239025+0000","last_clean":"2023-10-08T09:54:46.239025+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:54:46.239025+0000","last_undegraded":"2023-10-08T09:54:46.239025+0000","last_fullsized":"2023-10-08T09:54:46.239025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275016+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:54:46.275016+0000","last_peered":"2023-10-08T09:54:46.275016+0000","last_clean":"2023-10-08T09:54:46.275016+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:54:46.275016+0000","last_undegraded":"2023-10-08T09:54:46.275016+0000","last_fullsized":"2023-10-08T09:54:46.275016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321405+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:55:46.321405+0000","last_peered":"2023-10-08T09:55:46.321405+0000","last_clean":"2023-10-08T09:55:46.321405+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:55:46.321405+0000","last_undegraded":"2023-10-08T09:55:46.321405+0000","last_fullsized":"2023-10-08T09:55:46.321405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245062+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:54:47.245062+0000","last_peered":"2023-10-08T09:54:47.245062+0000","last_clean":"2023-10-08T09:54:47.245062+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:54:47.245062+0000","last_undegraded":"2023-10-08T09:54:47.245062+0000","last_fullsized":"2023-10-08T09:54:47.245062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275524+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:54:46.275524+0000","last_peered":"2023-10-08T09:54:46.275524+0000","last_clean":"2023-10-08T09:54:46.275524+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:54:46.275524+0000","last_undegraded":"2023-10-08T09:54:46.275524+0000","last_fullsized":"2023-10-08T09:54:46.275524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240708+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:54:46.240708+0000","last_peered":"2023-10-08T09:54:46.240708+0000","last_clean":"2023-10-08T09:54:46.240708+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:54:46.240708+0000","last_undegraded":"2023-10-08T09:54:46.240708+0000","last_fullsized":"2023-10-08T09:54:46.240708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242471+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:54:46.242471+0000","last_peered":"2023-10-08T09:54:46.242471+0000","last_clean":"2023-10-08T09:54:46.242471+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:54:46.242471+0000","last_undegraded":"2023-10-08T09:54:46.242471+0000","last_fullsized":"2023-10-08T09:54:46.242471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236665+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:54:47.236665+0000","last_peered":"2023-10-08T09:54:47.236665+0000","last_clean":"2023-10-08T09:54:47.236665+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:54:47.236665+0000","last_undegraded":"2023-10-08T09:54:47.236665+0000","last_fullsized":"2023-10-08T09:54:47.236665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236962+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:54:47.236962+0000","last_peered":"2023-10-08T09:54:47.236962+0000","last_clean":"2023-10-08T09:54:47.236962+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:54:47.236962+0000","last_undegraded":"2023-10-08T09:54:47.236962+0000","last_fullsized":"2023-10-08T09:54:47.236962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243351+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:54:47.243351+0000","last_peered":"2023-10-08T09:54:47.243351+0000","last_clean":"2023-10-08T09:54:47.243351+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:54:47.243351+0000","last_undegraded":"2023-10-08T09:54:47.243351+0000","last_fullsized":"2023-10-08T09:54:47.243351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242285+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:54:47.242285+0000","last_peered":"2023-10-08T09:54:47.242285+0000","last_clean":"2023-10-08T09:54:47.242285+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:54:47.242285+0000","last_undegraded":"2023-10-08T09:54:47.242285+0000","last_fullsized":"2023-10-08T09:54:47.242285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787058+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:55:46.787058+0000","last_peered":"2023-10-08T09:55:46.787058+0000","last_clean":"2023-10-08T09:55:46.787058+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:55:46.787058+0000","last_undegraded":"2023-10-08T09:55:46.787058+0000","last_fullsized":"2023-10-08T09:55:46.787058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786778+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:55:46.786778+0000","last_peered":"2023-10-08T09:55:46.786778+0000","last_clean":"2023-10-08T09:55:46.786778+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:55:46.786778+0000","last_undegraded":"2023-10-08T09:55:46.786778+0000","last_fullsized":"2023-10-08T09:55:46.786778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242460+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:54:47.242460+0000","last_peered":"2023-10-08T09:54:47.242460+0000","last_clean":"2023-10-08T09:54:47.242460+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:54:47.242460+0000","last_undegraded":"2023-10-08T09:54:47.242460+0000","last_fullsized":"2023-10-08T09:54:47.242460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321977+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:55:46.321977+0000","last_peered":"2023-10-08T09:55:46.321977+0000","last_clean":"2023-10-08T09:55:46.321977+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:55:46.321977+0000","last_undegraded":"2023-10-08T09:55:46.321977+0000","last_fullsized":"2023-10-08T09:55:46.321977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240292+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:54:46.240292+0000","last_peered":"2023-10-08T09:54:46.240292+0000","last_clean":"2023-10-08T09:54:46.240292+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:54:46.240292+0000","last_undegraded":"2023-10-08T09:54:46.240292+0000","last_fullsized":"2023-10-08T09:54:46.240292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322041+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:55:46.322041+0000","last_peered":"2023-10-08T09:55:46.322041+0000","last_clean":"2023-10-08T09:55:46.322041+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:55:46.322041+0000","last_undegraded":"2023-10-08T09:55:46.322041+0000","last_fullsized":"2023-10-08T09:55:46.322041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236960+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:54:47.236960+0000","last_peered":"2023-10-08T09:54:47.236960+0000","last_clean":"2023-10-08T09:54:47.236960+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:54:47.236960+0000","last_undegraded":"2023-10-08T09:54:47.236960+0000","last_fullsized":"2023-10-08T09:54:47.236960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240551+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:54:46.240551+0000","last_peered":"2023-10-08T09:54:46.240551+0000","last_clean":"2023-10-08T09:54:46.240551+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:54:46.240551+0000","last_undegraded":"2023-10-08T09:54:46.240551+0000","last_fullsized":"2023-10-08T09:54:46.240551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.237005+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:54:46.237005+0000","last_peered":"2023-10-08T09:54:46.237005+0000","last_clean":"2023-10-08T09:54:46.237005+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:54:46.237005+0000","last_undegraded":"2023-10-08T09:54:46.237005+0000","last_fullsized":"2023-10-08T09:54:46.237005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240404+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:54:46.240404+0000","last_peered":"2023-10-08T09:54:46.240404+0000","last_clean":"2023-10-08T09:54:46.240404+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:54:46.240404+0000","last_undegraded":"2023-10-08T09:54:46.240404+0000","last_fullsized":"2023-10-08T09:54:46.240404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235820+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:54:46.235820+0000","last_peered":"2023-10-08T09:54:46.235820+0000","last_clean":"2023-10-08T09:54:46.235820+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:54:46.235820+0000","last_undegraded":"2023-10-08T09:54:46.235820+0000","last_fullsized":"2023-10-08T09:54:46.235820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786977+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:55:46.786977+0000","last_peered":"2023-10-08T09:55:46.786977+0000","last_clean":"2023-10-08T09:55:46.786977+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:55:46.786977+0000","last_undegraded":"2023-10-08T09:55:46.786977+0000","last_fullsized":"2023-10-08T09:55:46.786977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.310191+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:43.310191+0000","last_peered":"2023-10-08T09:55:43.310191+0000","last_clean":"2023-10-08T09:55:43.310191+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:43.310191+0000","last_undegraded":"2023-10-08T09:55:43.310191+0000","last_fullsized":"2023-10-08T09:55:43.310191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235625+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:54:46.235625+0000","last_peered":"2023-10-08T09:54:46.235625+0000","last_clean":"2023-10-08T09:54:46.235625+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:54:46.235625+0000","last_undegraded":"2023-10-08T09:54:46.235625+0000","last_fullsized":"2023-10-08T09:54:46.235625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786805+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:55:46.786805+0000","last_peered":"2023-10-08T09:55:46.786805+0000","last_clean":"2023-10-08T09:55:46.786805+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:55:46.786805+0000","last_undegraded":"2023-10-08T09:55:46.786805+0000","last_fullsized":"2023-10-08T09:55:46.786805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.238970+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:54:46.238970+0000","last_peered":"2023-10-08T09:54:46.238970+0000","last_clean":"2023-10-08T09:54:46.238970+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:54:46.238970+0000","last_undegraded":"2023-10-08T09:54:46.238970+0000","last_fullsized":"2023-10-08T09:54:46.238970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.243621+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:54:46.243621+0000","last_peered":"2023-10-08T09:54:46.243621+0000","last_clean":"2023-10-08T09:54:46.243621+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:54:46.243621+0000","last_undegraded":"2023-10-08T09:54:46.243621+0000","last_fullsized":"2023-10-08T09:54:46.243621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243249+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:54:47.243249+0000","last_peered":"2023-10-08T09:54:47.243249+0000","last_clean":"2023-10-08T09:54:47.243249+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:54:47.243249+0000","last_undegraded":"2023-10-08T09:54:47.243249+0000","last_fullsized":"2023-10-08T09:54:47.243249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241824+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:54:46.241824+0000","last_peered":"2023-10-08T09:54:46.241824+0000","last_clean":"2023-10-08T09:54:46.241824+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:54:46.241824+0000","last_undegraded":"2023-10-08T09:54:46.241824+0000","last_fullsized":"2023-10-08T09:54:46.241824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321353+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:55:46.321353+0000","last_peered":"2023-10-08T09:55:46.321353+0000","last_clean":"2023-10-08T09:55:46.321353+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:55:46.321353+0000","last_undegraded":"2023-10-08T09:55:46.321353+0000","last_fullsized":"2023-10-08T09:55:46.321353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236698+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:54:47.236698+0000","last_peered":"2023-10-08T09:54:47.236698+0000","last_clean":"2023-10-08T09:54:47.236698+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:54:47.236698+0000","last_undegraded":"2023-10-08T09:54:47.236698+0000","last_fullsized":"2023-10-08T09:54:47.236698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241151+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:54:46.241151+0000","last_peered":"2023-10-08T09:54:46.241151+0000","last_clean":"2023-10-08T09:54:46.241151+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:54:46.241151+0000","last_undegraded":"2023-10-08T09:54:46.241151+0000","last_fullsized":"2023-10-08T09:54:46.241151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242154+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:54:47.242154+0000","last_peered":"2023-10-08T09:54:47.242154+0000","last_clean":"2023-10-08T09:54:47.242154+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:54:47.242154+0000","last_undegraded":"2023-10-08T09:54:47.242154+0000","last_fullsized":"2023-10-08T09:54:47.242154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321535+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:55:46.321535+0000","last_peered":"2023-10-08T09:55:46.321535+0000","last_clean":"2023-10-08T09:55:46.321535+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:55:46.321535+0000","last_undegraded":"2023-10-08T09:55:46.321535+0000","last_fullsized":"2023-10-08T09:55:46.321535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.311163+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:44.311163+0000","last_peered":"2023-10-08T09:55:44.311163+0000","last_clean":"2023-10-08T09:55:44.311163+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:44.311163+0000","last_undegraded":"2023-10-08T09:55:44.311163+0000","last_fullsized":"2023-10-08T09:55:44.311163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.241714+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:54:47.241714+0000","last_peered":"2023-10-08T09:54:47.241714+0000","last_clean":"2023-10-08T09:54:47.241714+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:54:47.241714+0000","last_undegraded":"2023-10-08T09:54:47.241714+0000","last_fullsized":"2023-10-08T09:54:47.241714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275440+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:54:46.275440+0000","last_peered":"2023-10-08T09:54:46.275440+0000","last_clean":"2023-10-08T09:54:46.275440+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:54:46.275440+0000","last_undegraded":"2023-10-08T09:54:46.275440+0000","last_fullsized":"2023-10-08T09:54:46.275440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236932+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:54:47.236932+0000","last_peered":"2023-10-08T09:54:47.236932+0000","last_clean":"2023-10-08T09:54:47.236932+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:54:47.236932+0000","last_undegraded":"2023-10-08T09:54:47.236932+0000","last_fullsized":"2023-10-08T09:54:47.236932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240734+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:54:46.240734+0000","last_peered":"2023-10-08T09:54:46.240734+0000","last_clean":"2023-10-08T09:54:46.240734+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:54:46.240734+0000","last_undegraded":"2023-10-08T09:54:46.240734+0000","last_fullsized":"2023-10-08T09:54:46.240734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242006+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:54:46.242006+0000","last_peered":"2023-10-08T09:54:46.242006+0000","last_clean":"2023-10-08T09:54:46.242006+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:54:46.242006+0000","last_undegraded":"2023-10-08T09:54:46.242006+0000","last_fullsized":"2023-10-08T09:54:46.242006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242039+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:54:47.242039+0000","last_peered":"2023-10-08T09:54:47.242039+0000","last_clean":"2023-10-08T09:54:47.242039+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:54:47.242039+0000","last_undegraded":"2023-10-08T09:54:47.242039+0000","last_fullsized":"2023-10-08T09:54:47.242039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":220,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:45.330927+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:45.330927+0000","last_peered":"2023-10-08T09:55:45.330927+0000","last_clean":"2023-10-08T09:55:45.330927+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:45.330927+0000","last_undegraded":"2023-10-08T09:55:45.330927+0000","last_fullsized":"2023-10-08T09:55:45.330927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":85,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235828+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:54:46.235828+0000","last_peered":"2023-10-08T09:54:46.235828+0000","last_clean":"2023-10-08T09:54:46.235828+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:54:46.235828+0000","last_undegraded":"2023-10-08T09:54:46.235828+0000","last_fullsized":"2023-10-08T09:54:46.235828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.372063+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:54:47.372063+0000","last_peered":"2023-10-08T09:54:47.372063+0000","last_clean":"2023-10-08T09:54:47.372063+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:54:47.372063+0000","last_undegraded":"2023-10-08T09:54:47.372063+0000","last_fullsized":"2023-10-08T09:54:47.372063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786794+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:55:46.786794+0000","last_peered":"2023-10-08T09:55:46.786794+0000","last_clean":"2023-10-08T09:55:46.786794+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:55:46.786794+0000","last_undegraded":"2023-10-08T09:55:46.786794+0000","last_fullsized":"2023-10-08T09:55:46.786794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241726+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:54:46.241726+0000","last_peered":"2023-10-08T09:54:46.241726+0000","last_clean":"2023-10-08T09:54:46.241726+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:54:46.241726+0000","last_undegraded":"2023-10-08T09:54:46.241726+0000","last_fullsized":"2023-10-08T09:54:46.241726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242355+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:54:47.242355+0000","last_peered":"2023-10-08T09:54:47.242355+0000","last_clean":"2023-10-08T09:54:47.242355+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:54:47.242355+0000","last_undegraded":"2023-10-08T09:54:47.242355+0000","last_fullsized":"2023-10-08T09:54:47.242355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.244247+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:54:46.244247+0000","last_peered":"2023-10-08T09:54:46.244247+0000","last_clean":"2023-10-08T09:54:46.244247+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:54:46.244247+0000","last_undegraded":"2023-10-08T09:54:46.244247+0000","last_fullsized":"2023-10-08T09:54:46.244247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275420+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:54:46.275420+0000","last_peered":"2023-10-08T09:54:46.275420+0000","last_clean":"2023-10-08T09:54:46.275420+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:54:46.275420+0000","last_undegraded":"2023-10-08T09:54:46.275420+0000","last_fullsized":"2023-10-08T09:54:46.275420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242227+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:54:47.242227+0000","last_peered":"2023-10-08T09:54:47.242227+0000","last_clean":"2023-10-08T09:54:47.242227+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:54:47.242227+0000","last_undegraded":"2023-10-08T09:54:47.242227+0000","last_fullsized":"2023-10-08T09:54:47.242227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:43.309721+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:43.309721+0000","last_peered":"2023-10-08T09:55:43.309721+0000","last_clean":"2023-10-08T09:55:43.309721+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:43.309721+0000","last_undegraded":"2023-10-08T09:55:43.309721+0000","last_fullsized":"2023-10-08T09:55:43.309721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275069+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:54:46.275069+0000","last_peered":"2023-10-08T09:54:46.275069+0000","last_clean":"2023-10-08T09:54:46.275069+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:54:46.275069+0000","last_undegraded":"2023-10-08T09:54:46.275069+0000","last_fullsized":"2023-10-08T09:54:46.275069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242040+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:54:46.242040+0000","last_peered":"2023-10-08T09:54:46.242040+0000","last_clean":"2023-10-08T09:54:46.242040+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:54:46.242040+0000","last_undegraded":"2023-10-08T09:54:46.242040+0000","last_fullsized":"2023-10-08T09:54:46.242040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.261776+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:54:47.261776+0000","last_peered":"2023-10-08T09:54:47.261776+0000","last_clean":"2023-10-08T09:54:47.261776+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:54:47.261776+0000","last_undegraded":"2023-10-08T09:54:47.261776+0000","last_fullsized":"2023-10-08T09:54:47.261776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241721+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:54:46.241721+0000","last_peered":"2023-10-08T09:54:46.241721+0000","last_clean":"2023-10-08T09:54:46.241721+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:54:46.241721+0000","last_undegraded":"2023-10-08T09:54:46.241721+0000","last_fullsized":"2023-10-08T09:54:46.241721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241621+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:54:46.241621+0000","last_peered":"2023-10-08T09:54:46.241621+0000","last_clean":"2023-10-08T09:54:46.241621+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:54:46.241621+0000","last_undegraded":"2023-10-08T09:54:46.241621+0000","last_fullsized":"2023-10-08T09:54:46.241621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.268332+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:54:47.268332+0000","last_peered":"2023-10-08T09:54:47.268332+0000","last_clean":"2023-10-08T09:54:47.268332+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:54:47.268332+0000","last_undegraded":"2023-10-08T09:54:47.268332+0000","last_fullsized":"2023-10-08T09:54:47.268332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242632+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:54:47.242632+0000","last_peered":"2023-10-08T09:54:47.242632+0000","last_clean":"2023-10-08T09:54:47.242632+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:54:47.242632+0000","last_undegraded":"2023-10-08T09:54:47.242632+0000","last_fullsized":"2023-10-08T09:54:47.242632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245612+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:54:47.245612+0000","last_peered":"2023-10-08T09:54:47.245612+0000","last_clean":"2023-10-08T09:54:47.245612+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:54:47.245612+0000","last_undegraded":"2023-10-08T09:54:47.245612+0000","last_fullsized":"2023-10-08T09:54:47.245612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242315+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:54:47.242315+0000","last_peered":"2023-10-08T09:54:47.242315+0000","last_clean":"2023-10-08T09:54:47.242315+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:54:47.242315+0000","last_undegraded":"2023-10-08T09:54:47.242315+0000","last_fullsized":"2023-10-08T09:54:47.242315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239430+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:54:46.239430+0000","last_peered":"2023-10-08T09:54:46.239430+0000","last_clean":"2023-10-08T09:54:46.239430+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:54:46.239430+0000","last_undegraded":"2023-10-08T09:54:46.239430+0000","last_fullsized":"2023-10-08T09:54:46.239430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241988+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:54:46.241988+0000","last_peered":"2023-10-08T09:54:46.241988+0000","last_clean":"2023-10-08T09:54:46.241988+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:54:46.241988+0000","last_undegraded":"2023-10-08T09:54:46.241988+0000","last_fullsized":"2023-10-08T09:54:46.241988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275187+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:54:46.275187+0000","last_peered":"2023-10-08T09:54:46.275187+0000","last_clean":"2023-10-08T09:54:46.275187+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:54:46.275187+0000","last_undegraded":"2023-10-08T09:54:46.275187+0000","last_fullsized":"2023-10-08T09:54:46.275187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.239044+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:54:46.239044+0000","last_peered":"2023-10-08T09:54:46.239044+0000","last_clean":"2023-10-08T09:54:46.239044+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:54:46.239044+0000","last_undegraded":"2023-10-08T09:54:46.239044+0000","last_fullsized":"2023-10-08T09:54:46.239044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242169+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:54:46.242169+0000","last_peered":"2023-10-08T09:54:46.242169+0000","last_clean":"2023-10-08T09:54:46.242169+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:54:46.242169+0000","last_undegraded":"2023-10-08T09:54:46.242169+0000","last_fullsized":"2023-10-08T09:54:46.242169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241532+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:54:46.241532+0000","last_peered":"2023-10-08T09:54:46.241532+0000","last_clean":"2023-10-08T09:54:46.241532+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:54:46.241532+0000","last_undegraded":"2023-10-08T09:54:46.241532+0000","last_fullsized":"2023-10-08T09:54:46.241532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:55:44.310385+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:44.310385+0000","last_peered":"2023-10-08T09:55:44.310385+0000","last_clean":"2023-10-08T09:55:44.310385+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:44.310385+0000","last_undegraded":"2023-10-08T09:55:44.310385+0000","last_fullsized":"2023-10-08T09:55:44.310385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275375+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:54:46.275375+0000","last_peered":"2023-10-08T09:54:46.275375+0000","last_clean":"2023-10-08T09:54:46.275375+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:54:46.275375+0000","last_undegraded":"2023-10-08T09:54:46.275375+0000","last_fullsized":"2023-10-08T09:54:46.275375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-10-08T09:53:56.877269+0000","last_change":"2023-10-08T09:55:46.965654+0000","last_active":"2023-10-08T09:53:56.178636+0000","last_peered":"2023-10-08T09:53:56.178636+0000","last_clean":"2023-10-08T09:53:56.178636+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T09:53:56.877269+0000","last_undegraded":"2023-10-08T09:53:56.877269+0000","last_fullsized":"2023-10-08T09:53:56.877269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240637+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:54:46.240637+0000","last_peered":"2023-10-08T09:54:46.240637+0000","last_clean":"2023-10-08T09:54:46.240637+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:54:46.240637+0000","last_undegraded":"2023-10-08T09:54:46.240637+0000","last_fullsized":"2023-10-08T09:54:46.240637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274935+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:54:46.274935+0000","last_peered":"2023-10-08T09:54:46.274935+0000","last_clean":"2023-10-08T09:54:46.274935+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:54:46.274935+0000","last_undegraded":"2023-10-08T09:54:46.274935+0000","last_fullsized":"2023-10-08T09:54:46.274935+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242154+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:54:46.242154+0000","last_peered":"2023-10-08T09:54:46.242154+0000","last_clean":"2023-10-08T09:54:46.242154+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:54:46.242154+0000","last_undegraded":"2023-10-08T09:54:46.242154+0000","last_fullsized":"2023-10-08T09:54:46.242154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.245430+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:54:47.245430+0000","last_peered":"2023-10-08T09:54:47.245430+0000","last_clean":"2023-10-08T09:54:47.245430+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:54:47.245430+0000","last_undegraded":"2023-10-08T09:54:47.245430+0000","last_fullsized":"2023-10-08T09:54:47.245430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.274957+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:54:46.274957+0000","last_peered":"2023-10-08T09:54:46.274957+0000","last_clean":"2023-10-08T09:54:46.274957+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:54:46.274957+0000","last_undegraded":"2023-10-08T09:54:46.274957+0000","last_fullsized":"2023-10-08T09:54:46.274957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242801+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:54:47.242801+0000","last_peered":"2023-10-08T09:54:47.242801+0000","last_clean":"2023-10-08T09:54:47.242801+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:54:47.242801+0000","last_undegraded":"2023-10-08T09:54:47.242801+0000","last_fullsized":"2023-10-08T09:54:47.242801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.264616+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:54:47.264616+0000","last_peered":"2023-10-08T09:54:47.264616+0000","last_clean":"2023-10-08T09:54:47.264616+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:54:47.264616+0000","last_undegraded":"2023-10-08T09:54:47.264616+0000","last_fullsized":"2023-10-08T09:54:47.264616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242093+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:54:46.242093+0000","last_peered":"2023-10-08T09:54:46.242093+0000","last_clean":"2023-10-08T09:54:46.242093+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:54:46.242093+0000","last_undegraded":"2023-10-08T09:54:46.242093+0000","last_fullsized":"2023-10-08T09:54:46.242093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321320+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:55:46.321320+0000","last_peered":"2023-10-08T09:55:46.321320+0000","last_clean":"2023-10-08T09:55:46.321320+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:55:46.321320+0000","last_undegraded":"2023-10-08T09:55:46.321320+0000","last_fullsized":"2023-10-08T09:55:46.321320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242196+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:54:47.242196+0000","last_peered":"2023-10-08T09:54:47.242196+0000","last_clean":"2023-10-08T09:54:47.242196+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:54:47.242196+0000","last_undegraded":"2023-10-08T09:54:47.242196+0000","last_fullsized":"2023-10-08T09:54:47.242196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321884+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:55:46.321884+0000","last_peered":"2023-10-08T09:55:46.321884+0000","last_clean":"2023-10-08T09:55:46.321884+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:55:46.321884+0000","last_undegraded":"2023-10-08T09:55:46.321884+0000","last_fullsized":"2023-10-08T09:55:46.321884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786506+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:55:46.786506+0000","last_peered":"2023-10-08T09:55:46.786506+0000","last_clean":"2023-10-08T09:55:46.786506+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:55:46.786506+0000","last_undegraded":"2023-10-08T09:55:46.786506+0000","last_fullsized":"2023-10-08T09:55:46.786506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241066+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:54:46.241066+0000","last_peered":"2023-10-08T09:54:46.241066+0000","last_clean":"2023-10-08T09:54:46.241066+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:54:46.241066+0000","last_undegraded":"2023-10-08T09:54:46.241066+0000","last_fullsized":"2023-10-08T09:54:46.241066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236862+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:54:47.236862+0000","last_peered":"2023-10-08T09:54:47.236862+0000","last_clean":"2023-10-08T09:54:47.236862+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:54:47.236862+0000","last_undegraded":"2023-10-08T09:54:47.236862+0000","last_fullsized":"2023-10-08T09:54:47.236862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240816+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:54:46.240816+0000","last_peered":"2023-10-08T09:54:46.240816+0000","last_clean":"2023-10-08T09:54:46.240816+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:54:46.240816+0000","last_undegraded":"2023-10-08T09:54:46.240816+0000","last_fullsized":"2023-10-08T09:54:46.240816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235394+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:54:46.235394+0000","last_peered":"2023-10-08T09:54:46.235394+0000","last_clean":"2023-10-08T09:54:46.235394+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:54:46.235394+0000","last_undegraded":"2023-10-08T09:54:46.235394+0000","last_fullsized":"2023-10-08T09:54:46.235394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.373098+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:54:47.373098+0000","last_peered":"2023-10-08T09:54:47.373098+0000","last_clean":"2023-10-08T09:54:47.373098+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:54:47.373098+0000","last_undegraded":"2023-10-08T09:54:47.373098+0000","last_fullsized":"2023-10-08T09:54:47.373098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321433+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:55:46.321433+0000","last_peered":"2023-10-08T09:55:46.321433+0000","last_clean":"2023-10-08T09:55:46.321433+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:55:46.321433+0000","last_undegraded":"2023-10-08T09:55:46.321433+0000","last_fullsized":"2023-10-08T09:55:46.321433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242374+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:54:47.242374+0000","last_peered":"2023-10-08T09:54:47.242374+0000","last_clean":"2023-10-08T09:54:47.242374+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:54:47.242374+0000","last_undegraded":"2023-10-08T09:54:47.242374+0000","last_fullsized":"2023-10-08T09:54:47.242374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241022+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:54:46.241022+0000","last_peered":"2023-10-08T09:54:46.241022+0000","last_clean":"2023-10-08T09:54:46.241022+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:54:46.241022+0000","last_undegraded":"2023-10-08T09:54:46.241022+0000","last_fullsized":"2023-10-08T09:54:46.241022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786643+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:55:46.786643+0000","last_peered":"2023-10-08T09:55:46.786643+0000","last_clean":"2023-10-08T09:55:46.786643+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:55:46.786643+0000","last_undegraded":"2023-10-08T09:55:46.786643+0000","last_fullsized":"2023-10-08T09:55:46.786643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321742+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:55:46.321742+0000","last_peered":"2023-10-08T09:55:46.321742+0000","last_clean":"2023-10-08T09:55:46.321742+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:55:46.321742+0000","last_undegraded":"2023-10-08T09:55:46.321742+0000","last_fullsized":"2023-10-08T09:55:46.321742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240976+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:54:46.240976+0000","last_peered":"2023-10-08T09:54:46.240976+0000","last_clean":"2023-10-08T09:54:46.240976+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:54:46.240976+0000","last_undegraded":"2023-10-08T09:54:46.240976+0000","last_fullsized":"2023-10-08T09:54:46.240976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.235467+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:54:46.235467+0000","last_peered":"2023-10-08T09:54:46.235467+0000","last_clean":"2023-10-08T09:54:46.235467+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:54:46.235467+0000","last_undegraded":"2023-10-08T09:54:46.235467+0000","last_fullsized":"2023-10-08T09:54:46.235467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275299+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:54:46.275299+0000","last_peered":"2023-10-08T09:54:46.275299+0000","last_clean":"2023-10-08T09:54:46.275299+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:54:46.275299+0000","last_undegraded":"2023-10-08T09:54:46.275299+0000","last_fullsized":"2023-10-08T09:54:46.275299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786917+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:55:46.786917+0000","last_peered":"2023-10-08T09:55:46.786917+0000","last_clean":"2023-10-08T09:55:46.786917+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:55:46.786917+0000","last_undegraded":"2023-10-08T09:55:46.786917+0000","last_fullsized":"2023-10-08T09:55:46.786917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275559+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:54:46.275559+0000","last_peered":"2023-10-08T09:54:46.275559+0000","last_clean":"2023-10-08T09:54:46.275559+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:54:46.275559+0000","last_undegraded":"2023-10-08T09:54:46.275559+0000","last_fullsized":"2023-10-08T09:54:46.275559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236370+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:54:47.236370+0000","last_peered":"2023-10-08T09:54:47.236370+0000","last_clean":"2023-10-08T09:54:47.236370+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:54:47.236370+0000","last_undegraded":"2023-10-08T09:54:47.236370+0000","last_fullsized":"2023-10-08T09:54:47.236370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242128+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:54:46.242128+0000","last_peered":"2023-10-08T09:54:46.242128+0000","last_clean":"2023-10-08T09:54:46.242128+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:54:46.242128+0000","last_undegraded":"2023-10-08T09:54:46.242128+0000","last_fullsized":"2023-10-08T09:54:46.242128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275239+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:54:46.275239+0000","last_peered":"2023-10-08T09:54:46.275239+0000","last_clean":"2023-10-08T09:54:46.275239+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:54:46.275239+0000","last_undegraded":"2023-10-08T09:54:46.275239+0000","last_fullsized":"2023-10-08T09:54:46.275239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.240860+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:54:46.240860+0000","last_peered":"2023-10-08T09:54:46.240860+0000","last_clean":"2023-10-08T09:54:46.240860+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:54:46.240860+0000","last_undegraded":"2023-10-08T09:54:46.240860+0000","last_fullsized":"2023-10-08T09:54:46.240860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241964+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:54:46.241964+0000","last_peered":"2023-10-08T09:54:46.241964+0000","last_clean":"2023-10-08T09:54:46.241964+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:54:46.241964+0000","last_undegraded":"2023-10-08T09:54:46.241964+0000","last_fullsized":"2023-10-08T09:54:46.241964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.275557+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:54:46.275557+0000","last_peered":"2023-10-08T09:54:46.275557+0000","last_clean":"2023-10-08T09:54:46.275557+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:54:46.275557+0000","last_undegraded":"2023-10-08T09:54:46.275557+0000","last_fullsized":"2023-10-08T09:54:46.275557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242860+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:54:47.242860+0000","last_peered":"2023-10-08T09:54:47.242860+0000","last_clean":"2023-10-08T09:54:47.242860+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:54:47.242860+0000","last_undegraded":"2023-10-08T09:54:47.242860+0000","last_fullsized":"2023-10-08T09:54:47.242860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786636+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:55:46.786636+0000","last_peered":"2023-10-08T09:55:46.786636+0000","last_clean":"2023-10-08T09:55:46.786636+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:55:46.786636+0000","last_undegraded":"2023-10-08T09:55:46.786636+0000","last_fullsized":"2023-10-08T09:55:46.786636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786645+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:55:46.786645+0000","last_peered":"2023-10-08T09:55:46.786645+0000","last_clean":"2023-10-08T09:55:46.786645+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:55:46.786645+0000","last_undegraded":"2023-10-08T09:55:46.786645+0000","last_fullsized":"2023-10-08T09:55:46.786645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.236564+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:54:47.236564+0000","last_peered":"2023-10-08T09:54:47.236564+0000","last_clean":"2023-10-08T09:54:47.236564+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:54:47.236564+0000","last_undegraded":"2023-10-08T09:54:47.236564+0000","last_fullsized":"2023-10-08T09:54:47.236564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241912+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:54:46.241912+0000","last_peered":"2023-10-08T09:54:46.241912+0000","last_clean":"2023-10-08T09:54:46.241912+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:54:46.241912+0000","last_undegraded":"2023-10-08T09:54:46.241912+0000","last_fullsized":"2023-10-08T09:54:46.241912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.243492+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:54:47.243492+0000","last_peered":"2023-10-08T09:54:47.243492+0000","last_clean":"2023-10-08T09:54:47.243492+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:54:47.243492+0000","last_undegraded":"2023-10-08T09:54:47.243492+0000","last_fullsized":"2023-10-08T09:54:47.243492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787148+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:55:46.787148+0000","last_peered":"2023-10-08T09:55:46.787148+0000","last_clean":"2023-10-08T09:55:46.787148+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:55:46.787148+0000","last_undegraded":"2023-10-08T09:55:46.787148+0000","last_fullsized":"2023-10-08T09:55:46.787148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321983+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:55:46.321983+0000","last_peered":"2023-10-08T09:55:46.321983+0000","last_clean":"2023-10-08T09:55:46.321983+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:55:46.321983+0000","last_undegraded":"2023-10-08T09:55:46.321983+0000","last_fullsized":"2023-10-08T09:55:46.321983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.241050+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:54:46.241050+0000","last_peered":"2023-10-08T09:54:46.241050+0000","last_clean":"2023-10-08T09:54:46.241050+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:54:46.241050+0000","last_undegraded":"2023-10-08T09:54:46.241050+0000","last_fullsized":"2023-10-08T09:54:46.241050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321400+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:55:46.321400+0000","last_peered":"2023-10-08T09:55:46.321400+0000","last_clean":"2023-10-08T09:55:46.321400+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:55:46.321400+0000","last_undegraded":"2023-10-08T09:55:46.321400+0000","last_fullsized":"2023-10-08T09:55:46.321400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:54:46.242613+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:54:46.242613+0000","last_peered":"2023-10-08T09:54:46.242613+0000","last_clean":"2023-10-08T09:54:46.242613+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:54:46.242613+0000","last_undegraded":"2023-10-08T09:54:46.242613+0000","last_fullsized":"2023-10-08T09:54:46.242613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:54:47.242750+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:54:47.242750+0000","last_peered":"2023-10-08T09:54:47.242750+0000","last_clean":"2023-10-08T09:54:47.242750+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:54:47.242750+0000","last_undegraded":"2023-10-08T09:54:47.242750+0000","last_fullsized":"2023-10-08T09:54:47.242750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":265,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":485589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.73899999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":0.879},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.85099999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.755},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.80400000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1071443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.96999999999999997}]}]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.623},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.746}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78100000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81799999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.84699999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68300000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.56699999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.745}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.503},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.90000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0049999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.77400000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.81699999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1067555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59399999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85499999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.82399999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.92100000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.90500000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.49099999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.623},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.61099999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.226,"15min":1.226},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93600000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.73099999999999998}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":489177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67600000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.878}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80500000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.95999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":476873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.27300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.315}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35199999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.44900000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46700000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.60899999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":495777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.88500000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.94299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.92700000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.008}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54000000000000004,"5min":0.621,"15min":0.621},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"min":{"1min":0.27400000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":1.3959999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82599999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":1.919}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":4.1840000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":5.0140000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.99199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":5.8659999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5170000000000003,"5min":6.5170000000000003,"15min":6.5170000000000003},"last":6.5170000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.1279999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":7.2649999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":8.1259999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.014}]}]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":479881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45700000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55300000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.78500000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.71399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.77100000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1589248,"data_stored":1067239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.48799999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:51.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:51 smithi097 ceph-mon[99491]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:51 smithi097 ceph-mon[99491]: from='client.15621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:51 smithi097 ceph-mon[99491]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2023-10-08T09:55:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:51 smithi097 ceph-mon[99491]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:51 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2724710928' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-08T09:55:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:51 smithi178 ceph-mon[103156]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:51 smithi178 ceph-mon[103156]: from='client.15621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:51 smithi178 ceph-mon[103156]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2023-10-08T09:55:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:51 smithi178 ceph-mon[103156]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:51 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2724710928' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-08T09:55:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:51 smithi186 ceph-mon[104393]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:51 smithi186 ceph-mon[104393]: from='client.15621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:51 smithi186 ceph-mon[104393]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2023-10-08T09:55:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:51 smithi186 ceph-mon[104393]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:51 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2724710928' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-08T09:55:52.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:52 smithi097 ceph-mon[99491]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:52.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:52 smithi178 ceph-mon[103156]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:52.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:52 smithi186 ceph-mon[104393]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:53.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:53 smithi097 ceph-mon[99491]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:53.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:53 smithi178 ceph-mon[103156]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:53 smithi186 ceph-mon[104393]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:55:54.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:54.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.230+0000 7f578956a700 1 -- 172.21.15.97:0/1266048576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 msgr2=0x7f5784100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.230+0000 7f578956a700 1 --2- 172.21.15.97:0/1266048576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784100c20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f57780099c0 tx=0x7f5778009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 -- 172.21.15.97:0/1266048576 shutdown_connections 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 --2- 172.21.15.97:0/1266048576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 0x7f5784107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 --2- 172.21.15.97:0/1266048576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f57840ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 --2- 172.21.15.97:0/1266048576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784100c20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 -- 172.21.15.97:0/1266048576 >> 172.21.15.97:0/1266048576 conn(0x7f57840fab40 msgr2=0x7f57840fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 -- 172.21.15.97:0/1266048576 shutdown_connections 2023-10-08T09:55:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 -- 172.21.15.97:0/1266048576 wait complete. 2023-10-08T09:55:54.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.231+0000 7f578956a700 1 Processor -- start 2023-10-08T09:55:54.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 -- start start 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f5784195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 0x7f578419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578419d060 con 0x7f57841007c0 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578419d1a0 con 0x7f57840ff5d0 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f578956a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578419d330 con 0x7f5784101160 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57040/0 (socket says 172.21.15.97:57040) 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 -- 172.21.15.97:0/2294644997 learned_addr learned my addr 172.21.15.97:0/2294644997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f5782ffd700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f5784195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 -- 172.21.15.97:0/2294644997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 msgr2=0x7f578419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57837fe700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 0x7f578419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 0x7f578419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 -- 172.21.15.97:0/2294644997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 msgr2=0x7f5784195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f5784195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 -- 172.21.15.97:0/2294644997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780095f0 con 0x7f57841007c0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f5782ffd700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f5784195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.232+0000 7f57827fc700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5778003730 tx=0x7f577800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.233+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577801d070 con 0x7f57841007c0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.233+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f577800bda0 con 0x7f57841007c0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.233+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578419d5b0 con 0x7f57841007c0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.233+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778017650 con 0x7f57841007c0 2023-10-08T09:55:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.233+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578419d9d0 con 0x7f57841007c0 2023-10-08T09:55:54.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.234+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57841069a0 con 0x7f57841007c0 2023-10-08T09:55:54.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.235+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5778022470 con 0x7f57841007c0 2023-10-08T09:55:54.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.235+0000 7f576bfff700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 0x7f576c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.235+0000 7f5782ffd700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 0x7f576c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.235+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f57780ce310 con 0x7f57841007c0 2023-10-08T09:55:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.236+0000 7f5782ffd700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 0x7f576c0a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5784100620 tx=0x7f5774009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.240+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f57780976d0 con 0x7f57841007c0 2023-10-08T09:55:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.428+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f578402d070 con 0x7f576c0a1480 2023-10-08T09:55:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.439+0000 7f576bfff700 1 -- 172.21.15.97:0/2294644997 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427361 (secure 0 0 0) 0x7f578402d070 con 0x7f576c0a1480 2023-10-08T09:55:54.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 msgr2=0x7f576c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 0x7f576c0a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5784100620 tx=0x7f5774009670 comp rx=0 tx=0).stop 2023-10-08T09:55:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 msgr2=0x7f5784195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5778003730 tx=0x7f577800fab0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 shutdown_connections 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f576c0a1480 0x7f576c0a3940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5784101160 0x7f578419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57840ff5d0 0x7f5784195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 --2- 172.21.15.97:0/2294644997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57841007c0 0x7f5784195a70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 >> 172.21.15.97:0/2294644997 conn(0x7f57840fab40 msgr2=0x7f5784103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 shutdown_connections 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.443+0000 7f578956a700 1 -- 172.21.15.97:0/2294644997 wait complete. 2023-10-08T09:55:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:54.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:54 smithi097 ceph-mon[99491]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-08T09:55:54.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:54 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T09:55:54.458 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-10-08T09:55:53.909108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":659,"num_read_kb":389,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337964,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603860,"statfs":{"total":1151940427776,"available":1151594352640,"internally_reserved":0,"allocated":14020608,"data_stored":7728708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":26,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002515"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256331+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:55:46.256331+0000","last_peered":"2023-10-08T09:55:46.256331+0000","last_clean":"2023-10-08T09:55:46.256331+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:55:46.256331+0000","last_undegraded":"2023-10-08T09:55:46.256331+0000","last_fullsized":"2023-10-08T09:55:46.256331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145930+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:55:47.145930+0000","last_peered":"2023-10-08T09:55:47.145930+0000","last_clean":"2023-10-08T09:55:47.145930+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:55:47.145930+0000","last_undegraded":"2023-10-08T09:55:47.145930+0000","last_fullsized":"2023-10-08T09:55:47.145930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145702+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:55:47.145702+0000","last_peered":"2023-10-08T09:55:47.145702+0000","last_clean":"2023-10-08T09:55:47.145702+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:55:47.145702+0000","last_undegraded":"2023-10-08T09:55:47.145702+0000","last_fullsized":"2023-10-08T09:55:47.145702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317562+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:55:46.317562+0000","last_peered":"2023-10-08T09:55:46.317562+0000","last_clean":"2023-10-08T09:55:46.317562+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:55:46.317562+0000","last_undegraded":"2023-10-08T09:55:46.317562+0000","last_fullsized":"2023-10-08T09:55:46.317562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721987+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:55:46.721987+0000","last_peered":"2023-10-08T09:55:46.721987+0000","last_clean":"2023-10-08T09:55:46.721987+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:55:46.721987+0000","last_undegraded":"2023-10-08T09:55:46.721987+0000","last_fullsized":"2023-10-08T09:55:46.721987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315991+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:55:46.315991+0000","last_peered":"2023-10-08T09:55:46.315991+0000","last_clean":"2023-10-08T09:55:46.315991+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:55:46.315991+0000","last_undegraded":"2023-10-08T09:55:46.315991+0000","last_fullsized":"2023-10-08T09:55:46.315991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253800+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:55:46.253800+0000","last_peered":"2023-10-08T09:55:46.253800+0000","last_clean":"2023-10-08T09:55:46.253800+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:55:46.253800+0000","last_undegraded":"2023-10-08T09:55:46.253800+0000","last_fullsized":"2023-10-08T09:55:46.253800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321405+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:55:46.321405+0000","last_peered":"2023-10-08T09:55:46.321405+0000","last_clean":"2023-10-08T09:55:46.321405+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:55:46.321405+0000","last_undegraded":"2023-10-08T09:55:46.321405+0000","last_fullsized":"2023-10-08T09:55:46.321405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316356+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:55:46.316356+0000","last_peered":"2023-10-08T09:55:46.316356+0000","last_clean":"2023-10-08T09:55:46.316356+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:55:46.316356+0000","last_undegraded":"2023-10-08T09:55:46.316356+0000","last_fullsized":"2023-10-08T09:55:46.316356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254208+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:55:46.254208+0000","last_peered":"2023-10-08T09:55:46.254208+0000","last_clean":"2023-10-08T09:55:46.254208+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:55:46.254208+0000","last_undegraded":"2023-10-08T09:55:46.254208+0000","last_fullsized":"2023-10-08T09:55:46.254208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317433+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:55:46.317433+0000","last_peered":"2023-10-08T09:55:46.317433+0000","last_clean":"2023-10-08T09:55:46.317433+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:55:46.317433+0000","last_undegraded":"2023-10-08T09:55:46.317433+0000","last_fullsized":"2023-10-08T09:55:46.317433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315009+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:55:46.315009+0000","last_peered":"2023-10-08T09:55:46.315009+0000","last_clean":"2023-10-08T09:55:46.315009+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:55:46.315009+0000","last_undegraded":"2023-10-08T09:55:46.315009+0000","last_fullsized":"2023-10-08T09:55:46.315009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145676+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:55:47.145676+0000","last_peered":"2023-10-08T09:55:47.145676+0000","last_clean":"2023-10-08T09:55:47.145676+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:55:47.145676+0000","last_undegraded":"2023-10-08T09:55:47.145676+0000","last_fullsized":"2023-10-08T09:55:47.145676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145948+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:55:47.145948+0000","last_peered":"2023-10-08T09:55:47.145948+0000","last_clean":"2023-10-08T09:55:47.145948+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:55:47.145948+0000","last_undegraded":"2023-10-08T09:55:47.145948+0000","last_fullsized":"2023-10-08T09:55:47.145948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256320+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:55:46.256320+0000","last_peered":"2023-10-08T09:55:46.256320+0000","last_clean":"2023-10-08T09:55:46.256320+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:55:46.256320+0000","last_undegraded":"2023-10-08T09:55:46.256320+0000","last_fullsized":"2023-10-08T09:55:46.256320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253841+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:55:46.253841+0000","last_peered":"2023-10-08T09:55:46.253841+0000","last_clean":"2023-10-08T09:55:46.253841+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:55:46.253841+0000","last_undegraded":"2023-10-08T09:55:46.253841+0000","last_fullsized":"2023-10-08T09:55:46.253841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787058+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:55:46.787058+0000","last_peered":"2023-10-08T09:55:46.787058+0000","last_clean":"2023-10-08T09:55:46.787058+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:55:46.787058+0000","last_undegraded":"2023-10-08T09:55:46.787058+0000","last_fullsized":"2023-10-08T09:55:46.787058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786778+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:55:46.786778+0000","last_peered":"2023-10-08T09:55:46.786778+0000","last_clean":"2023-10-08T09:55:46.786778+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:55:46.786778+0000","last_undegraded":"2023-10-08T09:55:46.786778+0000","last_fullsized":"2023-10-08T09:55:46.786778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253805+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:55:46.253805+0000","last_peered":"2023-10-08T09:55:46.253805+0000","last_clean":"2023-10-08T09:55:46.253805+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:55:46.253805+0000","last_undegraded":"2023-10-08T09:55:46.253805+0000","last_fullsized":"2023-10-08T09:55:46.253805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321977+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:55:46.321977+0000","last_peered":"2023-10-08T09:55:46.321977+0000","last_clean":"2023-10-08T09:55:46.321977+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:55:46.321977+0000","last_undegraded":"2023-10-08T09:55:46.321977+0000","last_fullsized":"2023-10-08T09:55:46.321977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317139+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:55:46.317139+0000","last_peered":"2023-10-08T09:55:46.317139+0000","last_clean":"2023-10-08T09:55:46.317139+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:55:46.317139+0000","last_undegraded":"2023-10-08T09:55:46.317139+0000","last_fullsized":"2023-10-08T09:55:46.317139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322041+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:55:46.322041+0000","last_peered":"2023-10-08T09:55:46.322041+0000","last_clean":"2023-10-08T09:55:46.322041+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:55:46.322041+0000","last_undegraded":"2023-10-08T09:55:46.322041+0000","last_fullsized":"2023-10-08T09:55:46.322041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.146053+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:55:47.146053+0000","last_peered":"2023-10-08T09:55:47.146053+0000","last_clean":"2023-10-08T09:55:47.146053+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:55:47.146053+0000","last_undegraded":"2023-10-08T09:55:47.146053+0000","last_fullsized":"2023-10-08T09:55:47.146053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317320+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:55:46.317320+0000","last_peered":"2023-10-08T09:55:46.317320+0000","last_clean":"2023-10-08T09:55:46.317320+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:55:46.317320+0000","last_undegraded":"2023-10-08T09:55:46.317320+0000","last_fullsized":"2023-10-08T09:55:46.317320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722326+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:55:46.722326+0000","last_peered":"2023-10-08T09:55:46.722326+0000","last_clean":"2023-10-08T09:55:46.722326+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:55:46.722326+0000","last_undegraded":"2023-10-08T09:55:46.722326+0000","last_fullsized":"2023-10-08T09:55:46.722326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317255+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:55:46.317255+0000","last_peered":"2023-10-08T09:55:46.317255+0000","last_clean":"2023-10-08T09:55:46.317255+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:55:46.317255+0000","last_undegraded":"2023-10-08T09:55:46.317255+0000","last_fullsized":"2023-10-08T09:55:46.317255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722320+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:55:46.722320+0000","last_peered":"2023-10-08T09:55:46.722320+0000","last_clean":"2023-10-08T09:55:46.722320+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:55:46.722320+0000","last_undegraded":"2023-10-08T09:55:46.722320+0000","last_fullsized":"2023-10-08T09:55:46.722320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319415+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:55:46.319415+0000","last_peered":"2023-10-08T09:55:46.319415+0000","last_clean":"2023-10-08T09:55:46.319415+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:55:46.319415+0000","last_undegraded":"2023-10-08T09:55:46.319415+0000","last_fullsized":"2023-10-08T09:55:46.319415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786977+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:55:46.786977+0000","last_peered":"2023-10-08T09:55:46.786977+0000","last_clean":"2023-10-08T09:55:46.786977+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:55:46.786977+0000","last_undegraded":"2023-10-08T09:55:46.786977+0000","last_fullsized":"2023-10-08T09:55:46.786977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:48.325098+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:48.325098+0000","last_peered":"2023-10-08T09:55:48.325098+0000","last_clean":"2023-10-08T09:55:48.325098+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:48.325098+0000","last_undegraded":"2023-10-08T09:55:48.325098+0000","last_fullsized":"2023-10-08T09:55:48.325098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722196+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:55:46.722196+0000","last_peered":"2023-10-08T09:55:46.722196+0000","last_clean":"2023-10-08T09:55:46.722196+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:55:46.722196+0000","last_undegraded":"2023-10-08T09:55:46.722196+0000","last_fullsized":"2023-10-08T09:55:46.722196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786805+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:55:46.786805+0000","last_peered":"2023-10-08T09:55:46.786805+0000","last_clean":"2023-10-08T09:55:46.786805+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:55:46.786805+0000","last_undegraded":"2023-10-08T09:55:46.786805+0000","last_fullsized":"2023-10-08T09:55:46.786805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316010+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:55:46.316010+0000","last_peered":"2023-10-08T09:55:46.316010+0000","last_clean":"2023-10-08T09:55:46.316010+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:55:46.316010+0000","last_undegraded":"2023-10-08T09:55:46.316010+0000","last_fullsized":"2023-10-08T09:55:46.316010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319584+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:55:46.319584+0000","last_peered":"2023-10-08T09:55:46.319584+0000","last_clean":"2023-10-08T09:55:46.319584+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:55:46.319584+0000","last_undegraded":"2023-10-08T09:55:46.319584+0000","last_fullsized":"2023-10-08T09:55:46.319584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256222+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:55:46.256222+0000","last_peered":"2023-10-08T09:55:46.256222+0000","last_clean":"2023-10-08T09:55:46.256222+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:55:46.256222+0000","last_undegraded":"2023-10-08T09:55:46.256222+0000","last_fullsized":"2023-10-08T09:55:46.256222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314607+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:55:46.314607+0000","last_peered":"2023-10-08T09:55:46.314607+0000","last_clean":"2023-10-08T09:55:46.314607+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:55:46.314607+0000","last_undegraded":"2023-10-08T09:55:46.314607+0000","last_fullsized":"2023-10-08T09:55:46.314607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321353+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:55:46.321353+0000","last_peered":"2023-10-08T09:55:46.321353+0000","last_clean":"2023-10-08T09:55:46.321353+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:55:46.321353+0000","last_undegraded":"2023-10-08T09:55:46.321353+0000","last_fullsized":"2023-10-08T09:55:46.321353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145844+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:55:47.145844+0000","last_peered":"2023-10-08T09:55:47.145844+0000","last_clean":"2023-10-08T09:55:47.145844+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:55:47.145844+0000","last_undegraded":"2023-10-08T09:55:47.145844+0000","last_fullsized":"2023-10-08T09:55:47.145844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318886+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:55:46.318886+0000","last_peered":"2023-10-08T09:55:46.318886+0000","last_clean":"2023-10-08T09:55:46.318886+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:55:46.318886+0000","last_undegraded":"2023-10-08T09:55:46.318886+0000","last_fullsized":"2023-10-08T09:55:46.318886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253303+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:55:46.253303+0000","last_peered":"2023-10-08T09:55:46.253303+0000","last_clean":"2023-10-08T09:55:46.253303+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:55:46.253303+0000","last_undegraded":"2023-10-08T09:55:46.253303+0000","last_fullsized":"2023-10-08T09:55:46.253303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321535+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:55:46.321535+0000","last_peered":"2023-10-08T09:55:46.321535+0000","last_clean":"2023-10-08T09:55:46.321535+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:55:46.321535+0000","last_undegraded":"2023-10-08T09:55:46.321535+0000","last_fullsized":"2023-10-08T09:55:46.321535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:49.325875+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:49.325875+0000","last_peered":"2023-10-08T09:55:49.325875+0000","last_clean":"2023-10-08T09:55:49.325875+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:49.325875+0000","last_undegraded":"2023-10-08T09:55:49.325875+0000","last_fullsized":"2023-10-08T09:55:49.325875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253147+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:55:46.253147+0000","last_peered":"2023-10-08T09:55:46.253147+0000","last_clean":"2023-10-08T09:55:46.253147+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:55:46.253147+0000","last_undegraded":"2023-10-08T09:55:46.253147+0000","last_fullsized":"2023-10-08T09:55:46.253147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254260+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:55:46.254260+0000","last_peered":"2023-10-08T09:55:46.254260+0000","last_clean":"2023-10-08T09:55:46.254260+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:55:46.254260+0000","last_undegraded":"2023-10-08T09:55:46.254260+0000","last_fullsized":"2023-10-08T09:55:46.254260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145955+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:55:47.145955+0000","last_peered":"2023-10-08T09:55:47.145955+0000","last_clean":"2023-10-08T09:55:47.145955+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:55:47.145955+0000","last_undegraded":"2023-10-08T09:55:47.145955+0000","last_fullsized":"2023-10-08T09:55:47.145955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317450+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:55:46.317450+0000","last_peered":"2023-10-08T09:55:46.317450+0000","last_clean":"2023-10-08T09:55:46.317450+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:55:46.317450+0000","last_undegraded":"2023-10-08T09:55:46.317450+0000","last_fullsized":"2023-10-08T09:55:46.317450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319248+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:55:46.319248+0000","last_peered":"2023-10-08T09:55:46.319248+0000","last_clean":"2023-10-08T09:55:46.319248+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:55:46.319248+0000","last_undegraded":"2023-10-08T09:55:46.319248+0000","last_fullsized":"2023-10-08T09:55:46.319248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253282+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:55:46.253282+0000","last_peered":"2023-10-08T09:55:46.253282+0000","last_clean":"2023-10-08T09:55:46.253282+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:55:46.253282+0000","last_undegraded":"2023-10-08T09:55:46.253282+0000","last_fullsized":"2023-10-08T09:55:46.253282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":232,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:50.338454+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:50.338454+0000","last_peered":"2023-10-08T09:55:50.338454+0000","last_clean":"2023-10-08T09:55:50.338454+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:50.338454+0000","last_undegraded":"2023-10-08T09:55:50.338454+0000","last_fullsized":"2023-10-08T09:55:50.338454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":171,"num_read_kb":90,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722197+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:55:46.722197+0000","last_peered":"2023-10-08T09:55:46.722197+0000","last_clean":"2023-10-08T09:55:46.722197+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:55:46.722197+0000","last_undegraded":"2023-10-08T09:55:46.722197+0000","last_fullsized":"2023-10-08T09:55:46.722197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316087+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:55:46.316087+0000","last_peered":"2023-10-08T09:55:46.316087+0000","last_clean":"2023-10-08T09:55:46.316087+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:55:46.316087+0000","last_undegraded":"2023-10-08T09:55:46.316087+0000","last_fullsized":"2023-10-08T09:55:46.316087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786794+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:55:46.786794+0000","last_peered":"2023-10-08T09:55:46.786794+0000","last_clean":"2023-10-08T09:55:46.786794+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:55:46.786794+0000","last_undegraded":"2023-10-08T09:55:46.786794+0000","last_fullsized":"2023-10-08T09:55:46.786794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314846+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:55:46.314846+0000","last_peered":"2023-10-08T09:55:46.314846+0000","last_clean":"2023-10-08T09:55:46.314846+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:55:46.314846+0000","last_undegraded":"2023-10-08T09:55:46.314846+0000","last_fullsized":"2023-10-08T09:55:46.314846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255796+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:55:46.255796+0000","last_peered":"2023-10-08T09:55:46.255796+0000","last_clean":"2023-10-08T09:55:46.255796+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:55:46.255796+0000","last_undegraded":"2023-10-08T09:55:46.255796+0000","last_fullsized":"2023-10-08T09:55:46.255796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319462+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:55:46.319462+0000","last_peered":"2023-10-08T09:55:46.319462+0000","last_clean":"2023-10-08T09:55:46.319462+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:55:46.319462+0000","last_undegraded":"2023-10-08T09:55:46.319462+0000","last_fullsized":"2023-10-08T09:55:46.319462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254261+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:55:46.254261+0000","last_peered":"2023-10-08T09:55:46.254261+0000","last_clean":"2023-10-08T09:55:46.254261+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:55:46.254261+0000","last_undegraded":"2023-10-08T09:55:46.254261+0000","last_fullsized":"2023-10-08T09:55:46.254261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253707+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:55:46.253707+0000","last_peered":"2023-10-08T09:55:46.253707+0000","last_clean":"2023-10-08T09:55:46.253707+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:55:46.253707+0000","last_undegraded":"2023-10-08T09:55:46.253707+0000","last_fullsized":"2023-10-08T09:55:46.253707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":112,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:48.324136+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:48.324136+0000","last_peered":"2023-10-08T09:55:48.324136+0000","last_clean":"2023-10-08T09:55:48.324136+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:48.324136+0000","last_undegraded":"2023-10-08T09:55:48.324136+0000","last_fullsized":"2023-10-08T09:55:48.324136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253853+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:55:46.253853+0000","last_peered":"2023-10-08T09:55:46.253853+0000","last_clean":"2023-10-08T09:55:46.253853+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:55:46.253853+0000","last_undegraded":"2023-10-08T09:55:46.253853+0000","last_fullsized":"2023-10-08T09:55:46.253853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315051+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:55:46.315051+0000","last_peered":"2023-10-08T09:55:46.315051+0000","last_clean":"2023-10-08T09:55:46.315051+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:55:46.315051+0000","last_undegraded":"2023-10-08T09:55:46.315051+0000","last_fullsized":"2023-10-08T09:55:46.315051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721858+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:55:46.721858+0000","last_peered":"2023-10-08T09:55:46.721858+0000","last_clean":"2023-10-08T09:55:46.721858+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:55:46.721858+0000","last_undegraded":"2023-10-08T09:55:46.721858+0000","last_fullsized":"2023-10-08T09:55:46.721858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319008+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:55:46.319008+0000","last_peered":"2023-10-08T09:55:46.319008+0000","last_clean":"2023-10-08T09:55:46.319008+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:55:46.319008+0000","last_undegraded":"2023-10-08T09:55:46.319008+0000","last_fullsized":"2023-10-08T09:55:46.319008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318949+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:55:46.318949+0000","last_peered":"2023-10-08T09:55:46.318949+0000","last_clean":"2023-10-08T09:55:46.318949+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:55:46.318949+0000","last_undegraded":"2023-10-08T09:55:46.318949+0000","last_fullsized":"2023-10-08T09:55:46.318949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256224+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:55:46.256224+0000","last_peered":"2023-10-08T09:55:46.256224+0000","last_clean":"2023-10-08T09:55:46.256224+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:55:46.256224+0000","last_undegraded":"2023-10-08T09:55:46.256224+0000","last_fullsized":"2023-10-08T09:55:46.256224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256112+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:55:46.256112+0000","last_peered":"2023-10-08T09:55:46.256112+0000","last_clean":"2023-10-08T09:55:46.256112+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:55:46.256112+0000","last_undegraded":"2023-10-08T09:55:46.256112+0000","last_fullsized":"2023-10-08T09:55:46.256112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722119+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:55:46.722119+0000","last_peered":"2023-10-08T09:55:46.722119+0000","last_clean":"2023-10-08T09:55:46.722119+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:55:46.722119+0000","last_undegraded":"2023-10-08T09:55:46.722119+0000","last_fullsized":"2023-10-08T09:55:46.722119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253589+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:55:46.253589+0000","last_peered":"2023-10-08T09:55:46.253589+0000","last_clean":"2023-10-08T09:55:46.253589+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:55:46.253589+0000","last_undegraded":"2023-10-08T09:55:46.253589+0000","last_fullsized":"2023-10-08T09:55:46.253589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316199+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:55:46.316199+0000","last_peered":"2023-10-08T09:55:46.316199+0000","last_clean":"2023-10-08T09:55:46.316199+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:55:46.316199+0000","last_undegraded":"2023-10-08T09:55:46.316199+0000","last_fullsized":"2023-10-08T09:55:46.316199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314879+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:55:46.314879+0000","last_peered":"2023-10-08T09:55:46.314879+0000","last_clean":"2023-10-08T09:55:46.314879+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:55:46.314879+0000","last_undegraded":"2023-10-08T09:55:46.314879+0000","last_fullsized":"2023-10-08T09:55:46.314879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254174+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:55:46.254174+0000","last_peered":"2023-10-08T09:55:46.254174+0000","last_clean":"2023-10-08T09:55:46.254174+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:55:46.254174+0000","last_undegraded":"2023-10-08T09:55:46.254174+0000","last_fullsized":"2023-10-08T09:55:46.254174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316423+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:55:46.316423+0000","last_peered":"2023-10-08T09:55:46.316423+0000","last_clean":"2023-10-08T09:55:46.316423+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:55:46.316423+0000","last_undegraded":"2023-10-08T09:55:46.316423+0000","last_fullsized":"2023-10-08T09:55:46.316423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314834+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:55:46.314834+0000","last_peered":"2023-10-08T09:55:46.314834+0000","last_clean":"2023-10-08T09:55:46.314834+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:55:46.314834+0000","last_undegraded":"2023-10-08T09:55:46.314834+0000","last_fullsized":"2023-10-08T09:55:46.314834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319178+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:55:46.319178+0000","last_peered":"2023-10-08T09:55:46.319178+0000","last_clean":"2023-10-08T09:55:46.319178+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:55:46.319178+0000","last_undegraded":"2023-10-08T09:55:46.319178+0000","last_fullsized":"2023-10-08T09:55:46.319178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:49.324883+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:49.324883+0000","last_peered":"2023-10-08T09:55:49.324883+0000","last_clean":"2023-10-08T09:55:49.324883+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:49.324883+0000","last_undegraded":"2023-10-08T09:55:49.324883+0000","last_fullsized":"2023-10-08T09:55:49.324883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254261+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:55:46.254261+0000","last_peered":"2023-10-08T09:55:46.254261+0000","last_clean":"2023-10-08T09:55:46.254261+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:55:46.254261+0000","last_undegraded":"2023-10-08T09:55:46.254261+0000","last_fullsized":"2023-10-08T09:55:46.254261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.359149+0000","last_change":"2023-10-08T09:55:47.359149+0000","last_active":"2023-10-08T09:55:47.359149+0000","last_peered":"2023-10-08T09:55:47.359149+0000","last_clean":"2023-10-08T09:55:47.359149+0000","last_became_active":"2023-10-08T09:55:47.358812+0000","last_became_peered":"2023-10-08T09:55:47.358812+0000","last_unstale":"2023-10-08T09:55:47.359149+0000","last_undegraded":"2023-10-08T09:55:47.359149+0000","last_fullsized":"2023-10-08T09:55:47.359149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:21.157268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317403+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:55:46.317403+0000","last_peered":"2023-10-08T09:55:46.317403+0000","last_clean":"2023-10-08T09:55:46.317403+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:55:46.317403+0000","last_undegraded":"2023-10-08T09:55:46.317403+0000","last_fullsized":"2023-10-08T09:55:46.317403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253732+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:55:46.253732+0000","last_peered":"2023-10-08T09:55:46.253732+0000","last_clean":"2023-10-08T09:55:46.253732+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:55:46.253732+0000","last_undegraded":"2023-10-08T09:55:46.253732+0000","last_fullsized":"2023-10-08T09:55:46.253732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319385+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:55:46.319385+0000","last_peered":"2023-10-08T09:55:46.319385+0000","last_clean":"2023-10-08T09:55:46.319385+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:55:46.319385+0000","last_undegraded":"2023-10-08T09:55:46.319385+0000","last_fullsized":"2023-10-08T09:55:46.319385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314516+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:55:46.314516+0000","last_peered":"2023-10-08T09:55:46.314516+0000","last_clean":"2023-10-08T09:55:46.314516+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:55:46.314516+0000","last_undegraded":"2023-10-08T09:55:46.314516+0000","last_fullsized":"2023-10-08T09:55:46.314516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253790+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:55:46.253790+0000","last_peered":"2023-10-08T09:55:46.253790+0000","last_clean":"2023-10-08T09:55:46.253790+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:55:46.253790+0000","last_undegraded":"2023-10-08T09:55:46.253790+0000","last_fullsized":"2023-10-08T09:55:46.253790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255774+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:55:46.255774+0000","last_peered":"2023-10-08T09:55:46.255774+0000","last_clean":"2023-10-08T09:55:46.255774+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:55:46.255774+0000","last_undegraded":"2023-10-08T09:55:46.255774+0000","last_fullsized":"2023-10-08T09:55:46.255774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255918+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:55:46.255918+0000","last_peered":"2023-10-08T09:55:46.255918+0000","last_clean":"2023-10-08T09:55:46.255918+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:55:46.255918+0000","last_undegraded":"2023-10-08T09:55:46.255918+0000","last_fullsized":"2023-10-08T09:55:46.255918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314966+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:55:46.314966+0000","last_peered":"2023-10-08T09:55:46.314966+0000","last_clean":"2023-10-08T09:55:46.314966+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:55:46.314966+0000","last_undegraded":"2023-10-08T09:55:46.314966+0000","last_fullsized":"2023-10-08T09:55:46.314966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321320+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:55:46.321320+0000","last_peered":"2023-10-08T09:55:46.321320+0000","last_clean":"2023-10-08T09:55:46.321320+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:55:46.321320+0000","last_undegraded":"2023-10-08T09:55:46.321320+0000","last_fullsized":"2023-10-08T09:55:46.321320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253843+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:55:46.253843+0000","last_peered":"2023-10-08T09:55:46.253843+0000","last_clean":"2023-10-08T09:55:46.253843+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:55:46.253843+0000","last_undegraded":"2023-10-08T09:55:46.253843+0000","last_fullsized":"2023-10-08T09:55:46.253843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321884+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:55:46.321884+0000","last_peered":"2023-10-08T09:55:46.321884+0000","last_clean":"2023-10-08T09:55:46.321884+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:55:46.321884+0000","last_undegraded":"2023-10-08T09:55:46.321884+0000","last_fullsized":"2023-10-08T09:55:46.321884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786506+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:55:46.786506+0000","last_peered":"2023-10-08T09:55:46.786506+0000","last_clean":"2023-10-08T09:55:46.786506+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:55:46.786506+0000","last_undegraded":"2023-10-08T09:55:46.786506+0000","last_fullsized":"2023-10-08T09:55:46.786506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317840+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:55:46.317840+0000","last_peered":"2023-10-08T09:55:46.317840+0000","last_clean":"2023-10-08T09:55:46.317840+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:55:46.317840+0000","last_undegraded":"2023-10-08T09:55:46.317840+0000","last_fullsized":"2023-10-08T09:55:46.317840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145981+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:55:47.145981+0000","last_peered":"2023-10-08T09:55:47.145981+0000","last_clean":"2023-10-08T09:55:47.145981+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:55:47.145981+0000","last_undegraded":"2023-10-08T09:55:47.145981+0000","last_fullsized":"2023-10-08T09:55:47.145981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317664+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:55:46.317664+0000","last_peered":"2023-10-08T09:55:46.317664+0000","last_clean":"2023-10-08T09:55:46.317664+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:55:46.317664+0000","last_undegraded":"2023-10-08T09:55:46.317664+0000","last_fullsized":"2023-10-08T09:55:46.317664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721854+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:55:46.721854+0000","last_peered":"2023-10-08T09:55:46.721854+0000","last_clean":"2023-10-08T09:55:46.721854+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:55:46.721854+0000","last_undegraded":"2023-10-08T09:55:46.721854+0000","last_fullsized":"2023-10-08T09:55:46.721854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722292+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:55:46.722292+0000","last_peered":"2023-10-08T09:55:46.722292+0000","last_clean":"2023-10-08T09:55:46.722292+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:55:46.722292+0000","last_undegraded":"2023-10-08T09:55:46.722292+0000","last_fullsized":"2023-10-08T09:55:46.722292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321433+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:55:46.321433+0000","last_peered":"2023-10-08T09:55:46.321433+0000","last_clean":"2023-10-08T09:55:46.321433+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:55:46.321433+0000","last_undegraded":"2023-10-08T09:55:46.321433+0000","last_fullsized":"2023-10-08T09:55:46.321433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255634+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:55:46.255634+0000","last_peered":"2023-10-08T09:55:46.255634+0000","last_clean":"2023-10-08T09:55:46.255634+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:55:46.255634+0000","last_undegraded":"2023-10-08T09:55:46.255634+0000","last_fullsized":"2023-10-08T09:55:46.255634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317684+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:55:46.317684+0000","last_peered":"2023-10-08T09:55:46.317684+0000","last_clean":"2023-10-08T09:55:46.317684+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:55:46.317684+0000","last_undegraded":"2023-10-08T09:55:46.317684+0000","last_fullsized":"2023-10-08T09:55:46.317684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786643+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:55:46.786643+0000","last_peered":"2023-10-08T09:55:46.786643+0000","last_clean":"2023-10-08T09:55:46.786643+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:55:46.786643+0000","last_undegraded":"2023-10-08T09:55:46.786643+0000","last_fullsized":"2023-10-08T09:55:46.786643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321742+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:55:46.321742+0000","last_peered":"2023-10-08T09:55:46.321742+0000","last_clean":"2023-10-08T09:55:46.321742+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:55:46.321742+0000","last_undegraded":"2023-10-08T09:55:46.321742+0000","last_fullsized":"2023-10-08T09:55:46.321742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317547+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:55:46.317547+0000","last_peered":"2023-10-08T09:55:46.317547+0000","last_clean":"2023-10-08T09:55:46.317547+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:55:46.317547+0000","last_undegraded":"2023-10-08T09:55:46.317547+0000","last_fullsized":"2023-10-08T09:55:46.317547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722000+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:55:46.722000+0000","last_peered":"2023-10-08T09:55:46.722000+0000","last_clean":"2023-10-08T09:55:46.722000+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:55:46.722000+0000","last_undegraded":"2023-10-08T09:55:46.722000+0000","last_fullsized":"2023-10-08T09:55:46.722000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254319+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:55:46.254319+0000","last_peered":"2023-10-08T09:55:46.254319+0000","last_clean":"2023-10-08T09:55:46.254319+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:55:46.254319+0000","last_undegraded":"2023-10-08T09:55:46.254319+0000","last_fullsized":"2023-10-08T09:55:46.254319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786917+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:55:46.786917+0000","last_peered":"2023-10-08T09:55:46.786917+0000","last_clean":"2023-10-08T09:55:46.786917+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:55:46.786917+0000","last_undegraded":"2023-10-08T09:55:46.786917+0000","last_fullsized":"2023-10-08T09:55:46.786917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254333+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:55:46.254333+0000","last_peered":"2023-10-08T09:55:46.254333+0000","last_clean":"2023-10-08T09:55:46.254333+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:55:46.254333+0000","last_undegraded":"2023-10-08T09:55:46.254333+0000","last_fullsized":"2023-10-08T09:55:46.254333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145671+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:55:47.145671+0000","last_peered":"2023-10-08T09:55:47.145671+0000","last_clean":"2023-10-08T09:55:47.145671+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:55:47.145671+0000","last_undegraded":"2023-10-08T09:55:47.145671+0000","last_fullsized":"2023-10-08T09:55:47.145671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319384+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:55:46.319384+0000","last_peered":"2023-10-08T09:55:46.319384+0000","last_clean":"2023-10-08T09:55:46.319384+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:55:46.319384+0000","last_undegraded":"2023-10-08T09:55:46.319384+0000","last_fullsized":"2023-10-08T09:55:46.319384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254084+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:55:46.254084+0000","last_peered":"2023-10-08T09:55:46.254084+0000","last_clean":"2023-10-08T09:55:46.254084+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:55:46.254084+0000","last_undegraded":"2023-10-08T09:55:46.254084+0000","last_fullsized":"2023-10-08T09:55:46.254084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318763+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:55:46.318763+0000","last_peered":"2023-10-08T09:55:46.318763+0000","last_clean":"2023-10-08T09:55:46.318763+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:55:46.318763+0000","last_undegraded":"2023-10-08T09:55:46.318763+0000","last_fullsized":"2023-10-08T09:55:46.318763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315024+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:55:46.315024+0000","last_peered":"2023-10-08T09:55:46.315024+0000","last_clean":"2023-10-08T09:55:46.315024+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:55:46.315024+0000","last_undegraded":"2023-10-08T09:55:46.315024+0000","last_fullsized":"2023-10-08T09:55:46.315024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254297+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:55:46.254297+0000","last_peered":"2023-10-08T09:55:46.254297+0000","last_clean":"2023-10-08T09:55:46.254297+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:55:46.254297+0000","last_undegraded":"2023-10-08T09:55:46.254297+0000","last_fullsized":"2023-10-08T09:55:46.254297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255663+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:55:46.255663+0000","last_peered":"2023-10-08T09:55:46.255663+0000","last_clean":"2023-10-08T09:55:46.255663+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:55:46.255663+0000","last_undegraded":"2023-10-08T09:55:46.255663+0000","last_fullsized":"2023-10-08T09:55:46.255663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786636+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:55:46.786636+0000","last_peered":"2023-10-08T09:55:46.786636+0000","last_clean":"2023-10-08T09:55:46.786636+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:55:46.786636+0000","last_undegraded":"2023-10-08T09:55:46.786636+0000","last_fullsized":"2023-10-08T09:55:46.786636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786645+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:55:46.786645+0000","last_peered":"2023-10-08T09:55:46.786645+0000","last_clean":"2023-10-08T09:55:46.786645+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:55:46.786645+0000","last_undegraded":"2023-10-08T09:55:46.786645+0000","last_fullsized":"2023-10-08T09:55:46.786645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145836+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:55:47.145836+0000","last_peered":"2023-10-08T09:55:47.145836+0000","last_clean":"2023-10-08T09:55:47.145836+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:55:47.145836+0000","last_undegraded":"2023-10-08T09:55:47.145836+0000","last_fullsized":"2023-10-08T09:55:47.145836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314807+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:55:46.314807+0000","last_peered":"2023-10-08T09:55:46.314807+0000","last_clean":"2023-10-08T09:55:46.314807+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:55:46.314807+0000","last_undegraded":"2023-10-08T09:55:46.314807+0000","last_fullsized":"2023-10-08T09:55:46.314807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256045+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:55:46.256045+0000","last_peered":"2023-10-08T09:55:46.256045+0000","last_clean":"2023-10-08T09:55:46.256045+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:55:46.256045+0000","last_undegraded":"2023-10-08T09:55:46.256045+0000","last_fullsized":"2023-10-08T09:55:46.256045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787148+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:55:46.787148+0000","last_peered":"2023-10-08T09:55:46.787148+0000","last_clean":"2023-10-08T09:55:46.787148+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:55:46.787148+0000","last_undegraded":"2023-10-08T09:55:46.787148+0000","last_fullsized":"2023-10-08T09:55:46.787148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321983+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:55:46.321983+0000","last_peered":"2023-10-08T09:55:46.321983+0000","last_clean":"2023-10-08T09:55:46.321983+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:55:46.321983+0000","last_undegraded":"2023-10-08T09:55:46.321983+0000","last_fullsized":"2023-10-08T09:55:46.321983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318849+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:55:46.318849+0000","last_peered":"2023-10-08T09:55:46.318849+0000","last_clean":"2023-10-08T09:55:46.318849+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:55:46.318849+0000","last_undegraded":"2023-10-08T09:55:46.318849+0000","last_fullsized":"2023-10-08T09:55:46.318849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321400+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:55:46.321400+0000","last_peered":"2023-10-08T09:55:46.321400+0000","last_clean":"2023-10-08T09:55:46.321400+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:55:46.321400+0000","last_undegraded":"2023-10-08T09:55:46.321400+0000","last_fullsized":"2023-10-08T09:55:46.321400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315062+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:55:46.315062+0000","last_peered":"2023-10-08T09:55:46.315062+0000","last_clean":"2023-10-08T09:55:46.315062+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:55:46.315062+0000","last_undegraded":"2023-10-08T09:55:46.315062+0000","last_fullsized":"2023-10-08T09:55:46.315062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255827+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:55:46.255827+0000","last_peered":"2023-10-08T09:55:46.255827+0000","last_clean":"2023-10-08T09:55:46.255827+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:55:46.255827+0000","last_undegraded":"2023-10-08T09:55:46.255827+0000","last_fullsized":"2023-10-08T09:55:46.255827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":533,"num_read_kb":280,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":495165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.68500000000000005}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.86199999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.92500000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.96399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0109999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.98099999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1090587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41199999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67000000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58499999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74099999999999999}]}]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":495357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58499999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.308}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.43099999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46300000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56299999999999994}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.52700000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.62}]}]},{"osd":8,"up_from":56,"seq":240518168622,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.624}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97599999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.99199999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.88400000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76100000000000001,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.76400000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.052}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0229999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1086699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.378},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53800000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.69999999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.60499999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.752},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.628}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79900000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.74199999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":494629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.318}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.376},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59399999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63700000000000001,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67500000000000004,"5min":1.226,"15min":1.226},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67700000000000005,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81399999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.57499999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":498753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52700000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.376},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42099999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78800000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.749},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.80500000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73499999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":496017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.442}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.501},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65400000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60599999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.73799999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54500000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":495777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.88500000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.94299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.92700000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.008}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55500000000000005,"5min":0.621,"15min":0.621},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60499999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.309}]},{"osd":1,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56799999999999995,"15min":0.53400000000000003},"min":{"1min":0.438,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81499999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.442}]},{"osd":2,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.57499999999999996},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45400000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.625},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5170000000000003,"5min":6.5170000000000003,"15min":6.5170000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.60499999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58199999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64400000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55700000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.495}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.747},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.86299999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79600000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.84399999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.73399999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.90100000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78400000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.008},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.69999999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.97399999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.874},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:54.468 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-08T09:55:54.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:55:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:54 smithi186 ceph-mon[104393]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-08T09:55:54.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:54 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T09:55:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.616+0000 7f01bdd55700 1 -- 172.21.15.97:0/1392569006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 msgr2=0x7f01b8103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.616+0000 7f01bdd55700 1 --2- 172.21.15.97:0/1392569006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b8103580 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009990 tx=0x7f01ac009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.616+0000 7f01bdd55700 1 -- 172.21.15.97:0/1392569006 shutdown_connections 2023-10-08T09:55:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- 172.21.15.97:0/1392569006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b81063f0 0x7f01b8108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- 172.21.15.97:0/1392569006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b8103580 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- 172.21.15.97:0/1392569006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- 172.21.15.97:0/1392569006 >> 172.21.15.97:0/1392569006 conn(0x7f01b80fab40 msgr2=0x7f01b80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- 172.21.15.97:0/1392569006 shutdown_connections 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- 172.21.15.97:0/1392569006 wait complete. 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 Processor -- start 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- start start 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b81063f0 0x7f01b819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819e110 con 0x7f01b8101190 2023-10-08T09:55:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819e250 con 0x7f01b81063f0 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.617+0000 7f01bdd55700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819e3e0 con 0x7f01b8103ac0 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57052/0 (socket says 172.21.15.97:57052) 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 -- 172.21.15.97:0/4083212399 learned_addr learned my addr 172.21.15.97:0/4083212399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b6ffd700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 -- 172.21.15.97:0/4083212399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 msgr2=0x7f01b8196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 -- 172.21.15.97:0/4083212399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b81063f0 msgr2=0x7f01b819db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b81063f0 0x7f01b819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 -- 172.21.15.97:0/4083212399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0095f0 con 0x7f01b8101190 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b6ffd700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b77fe700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098e0 tx=0x7f01ac00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac01d070 con 0x7f01b8101190 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b819e660 con 0x7f01b8101190 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01ac00fca0 con 0x7f01b8101190 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac017810 con 0x7f01b8101190 2023-10-08T09:55:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.618+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b819eae0 con 0x7f01b8101190 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.619+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b80fc6c0 con 0x7f01b8101190 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.620+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f01ac022ac0 con 0x7f01b8101190 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.621+0000 7f01b4ff9700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 0x7f01a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.621+0000 7f01b6ffd700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 0x7f01a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.621+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01ac0ce9a0 con 0x7f01b8101190 2023-10-08T09:55:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.621+0000 7f01b6ffd700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 0x7f01a00a3940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005fd0 tx=0x7f01a8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.624+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f01ac097d60 con 0x7f01b8101190 2023-10-08T09:55:54.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:54 smithi178 ceph-mon[103156]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-08T09:55:54.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:54 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T09:55:54.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.811+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 --> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01b8060c30 con 0x7f01a00a1480 2023-10-08T09:55:54.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.824+0000 7f01b4ff9700 1 -- 172.21.15.97:0/4083212399 <== mgr.14152 v2:172.21.15.97:6800/3991091613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427361 (secure 0 0 0) 0x7f01b8060c30 con 0x7f01a00a1480 2023-10-08T09:55:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 msgr2=0x7f01a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 0x7f01a00a3940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005fd0 tx=0x7f01a8005dc0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 msgr2=0x7f01b81965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098e0 tx=0x7f01ac00f7b0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 shutdown_connections 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f01a00a1480 0x7f01a00a3940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b81063f0 0x7f01b819db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01b8101190 0x7f01b81965e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 --2- 172.21.15.97:0/4083212399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b8103ac0 0x7f01b8196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 >> 172.21.15.97:0/4083212399 conn(0x7f01b80fab40 msgr2=0x7f01b8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 shutdown_connections 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:54.828+0000 7f01bdd55700 1 -- 172.21.15.97:0/4083212399 wait complete. 2023-10-08T09:55:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-10-08T09:55:54.842 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-10-08T09:55:53.909108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598080,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":659,"num_read_kb":389,"num_write":312,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1184734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337964,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603860,"statfs":{"total":1151940427776,"available":1151594352640,"internally_reserved":0,"allocated":14020608,"data_stored":7728708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":260,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":26,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002515"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256331+0000","last_change":"2023-10-08T09:53:56.895827+0000","last_active":"2023-10-08T09:55:46.256331+0000","last_peered":"2023-10-08T09:55:46.256331+0000","last_clean":"2023-10-08T09:55:46.256331+0000","last_became_active":"2023-10-08T09:53:56.895441+0000","last_became_peered":"2023-10-08T09:53:56.895441+0000","last_unstale":"2023-10-08T09:55:46.256331+0000","last_undegraded":"2023-10-08T09:55:46.256331+0000","last_fullsized":"2023-10-08T09:55:46.256331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:32.493999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145930+0000","last_change":"2023-10-08T09:53:56.899469+0000","last_active":"2023-10-08T09:55:47.145930+0000","last_peered":"2023-10-08T09:55:47.145930+0000","last_clean":"2023-10-08T09:55:47.145930+0000","last_became_active":"2023-10-08T09:53:56.899230+0000","last_became_peered":"2023-10-08T09:53:56.899230+0000","last_unstale":"2023-10-08T09:55:47.145930+0000","last_undegraded":"2023-10-08T09:55:47.145930+0000","last_fullsized":"2023-10-08T09:55:47.145930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:01.289583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145702+0000","last_change":"2023-10-08T09:53:56.901507+0000","last_active":"2023-10-08T09:55:47.145702+0000","last_peered":"2023-10-08T09:55:47.145702+0000","last_clean":"2023-10-08T09:55:47.145702+0000","last_became_active":"2023-10-08T09:53:56.901184+0000","last_became_peered":"2023-10-08T09:53:56.901184+0000","last_unstale":"2023-10-08T09:55:47.145702+0000","last_undegraded":"2023-10-08T09:55:47.145702+0000","last_fullsized":"2023-10-08T09:55:47.145702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:40:50.910244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317562+0000","last_change":"2023-10-08T09:53:56.898403+0000","last_active":"2023-10-08T09:55:46.317562+0000","last_peered":"2023-10-08T09:55:46.317562+0000","last_clean":"2023-10-08T09:55:46.317562+0000","last_became_active":"2023-10-08T09:53:56.898090+0000","last_became_peered":"2023-10-08T09:53:56.898090+0000","last_unstale":"2023-10-08T09:55:46.317562+0000","last_undegraded":"2023-10-08T09:55:46.317562+0000","last_fullsized":"2023-10-08T09:55:46.317562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:02.704588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721987+0000","last_change":"2023-10-08T09:53:56.887176+0000","last_active":"2023-10-08T09:55:46.721987+0000","last_peered":"2023-10-08T09:55:46.721987+0000","last_clean":"2023-10-08T09:55:46.721987+0000","last_became_active":"2023-10-08T09:53:56.886741+0000","last_became_peered":"2023-10-08T09:53:56.886741+0000","last_unstale":"2023-10-08T09:55:46.721987+0000","last_undegraded":"2023-10-08T09:55:46.721987+0000","last_fullsized":"2023-10-08T09:55:46.721987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:12.669673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315991+0000","last_change":"2023-10-08T09:53:56.895033+0000","last_active":"2023-10-08T09:55:46.315991+0000","last_peered":"2023-10-08T09:55:46.315991+0000","last_clean":"2023-10-08T09:55:46.315991+0000","last_became_active":"2023-10-08T09:53:56.894661+0000","last_became_peered":"2023-10-08T09:53:56.894661+0000","last_unstale":"2023-10-08T09:55:46.315991+0000","last_undegraded":"2023-10-08T09:55:46.315991+0000","last_fullsized":"2023-10-08T09:55:46.315991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:56:49.630599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253800+0000","last_change":"2023-10-08T09:53:56.890247+0000","last_active":"2023-10-08T09:55:46.253800+0000","last_peered":"2023-10-08T09:55:46.253800+0000","last_clean":"2023-10-08T09:55:46.253800+0000","last_became_active":"2023-10-08T09:53:56.889931+0000","last_became_peered":"2023-10-08T09:53:56.889931+0000","last_unstale":"2023-10-08T09:55:46.253800+0000","last_undegraded":"2023-10-08T09:55:46.253800+0000","last_fullsized":"2023-10-08T09:55:46.253800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:38.746279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321405+0000","last_change":"2023-10-08T09:53:56.889794+0000","last_active":"2023-10-08T09:55:46.321405+0000","last_peered":"2023-10-08T09:55:46.321405+0000","last_clean":"2023-10-08T09:55:46.321405+0000","last_became_active":"2023-10-08T09:53:56.889451+0000","last_became_peered":"2023-10-08T09:53:56.889451+0000","last_unstale":"2023-10-08T09:55:46.321405+0000","last_undegraded":"2023-10-08T09:55:46.321405+0000","last_fullsized":"2023-10-08T09:55:46.321405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:14.255096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322392+0000","last_change":"2023-10-08T09:53:56.893523+0000","last_active":"2023-10-08T09:55:46.322392+0000","last_peered":"2023-10-08T09:55:46.322392+0000","last_clean":"2023-10-08T09:55:46.322392+0000","last_became_active":"2023-10-08T09:53:56.893022+0000","last_became_peered":"2023-10-08T09:53:56.893022+0000","last_unstale":"2023-10-08T09:55:46.322392+0000","last_undegraded":"2023-10-08T09:55:46.322392+0000","last_fullsized":"2023-10-08T09:55:46.322392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:18.938284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316356+0000","last_change":"2023-10-08T09:54:47.245062+0000","last_active":"2023-10-08T09:55:46.316356+0000","last_peered":"2023-10-08T09:55:46.316356+0000","last_clean":"2023-10-08T09:55:46.316356+0000","last_became_active":"2023-10-08T09:54:47.244679+0000","last_became_peered":"2023-10-08T09:54:47.244679+0000","last_unstale":"2023-10-08T09:55:46.316356+0000","last_undegraded":"2023-10-08T09:55:46.316356+0000","last_fullsized":"2023-10-08T09:55:46.316356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:34.644702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254208+0000","last_change":"2023-10-08T09:53:56.894750+0000","last_active":"2023-10-08T09:55:46.254208+0000","last_peered":"2023-10-08T09:55:46.254208+0000","last_clean":"2023-10-08T09:55:46.254208+0000","last_became_active":"2023-10-08T09:53:56.894448+0000","last_became_peered":"2023-10-08T09:53:56.894448+0000","last_unstale":"2023-10-08T09:55:46.254208+0000","last_undegraded":"2023-10-08T09:55:46.254208+0000","last_fullsized":"2023-10-08T09:55:46.254208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:45.411770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317433+0000","last_change":"2023-10-08T09:53:56.896044+0000","last_active":"2023-10-08T09:55:46.317433+0000","last_peered":"2023-10-08T09:55:46.317433+0000","last_clean":"2023-10-08T09:55:46.317433+0000","last_became_active":"2023-10-08T09:53:56.895002+0000","last_became_peered":"2023-10-08T09:53:56.895002+0000","last_unstale":"2023-10-08T09:55:46.317433+0000","last_undegraded":"2023-10-08T09:55:46.317433+0000","last_fullsized":"2023-10-08T09:55:46.317433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:09.754681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315009+0000","last_change":"2023-10-08T09:53:56.898898+0000","last_active":"2023-10-08T09:55:46.315009+0000","last_peered":"2023-10-08T09:55:46.315009+0000","last_clean":"2023-10-08T09:55:46.315009+0000","last_became_active":"2023-10-08T09:53:56.898564+0000","last_became_peered":"2023-10-08T09:53:56.898564+0000","last_unstale":"2023-10-08T09:55:46.315009+0000","last_undegraded":"2023-10-08T09:55:46.315009+0000","last_fullsized":"2023-10-08T09:55:46.315009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:34.953632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145676+0000","last_change":"2023-10-08T09:53:56.899178+0000","last_active":"2023-10-08T09:55:47.145676+0000","last_peered":"2023-10-08T09:55:47.145676+0000","last_clean":"2023-10-08T09:55:47.145676+0000","last_became_active":"2023-10-08T09:53:56.898792+0000","last_became_peered":"2023-10-08T09:53:56.898792+0000","last_unstale":"2023-10-08T09:55:47.145676+0000","last_undegraded":"2023-10-08T09:55:47.145676+0000","last_fullsized":"2023-10-08T09:55:47.145676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:57.708262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145948+0000","last_change":"2023-10-08T09:53:56.901622+0000","last_active":"2023-10-08T09:55:47.145948+0000","last_peered":"2023-10-08T09:55:47.145948+0000","last_clean":"2023-10-08T09:55:47.145948+0000","last_became_active":"2023-10-08T09:53:56.900716+0000","last_became_peered":"2023-10-08T09:53:56.900716+0000","last_unstale":"2023-10-08T09:55:47.145948+0000","last_undegraded":"2023-10-08T09:55:47.145948+0000","last_fullsized":"2023-10-08T09:55:47.145948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:38.007849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321933+0000","last_change":"2023-10-08T09:53:56.897222+0000","last_active":"2023-10-08T09:55:46.321933+0000","last_peered":"2023-10-08T09:55:46.321933+0000","last_clean":"2023-10-08T09:55:46.321933+0000","last_became_active":"2023-10-08T09:53:56.896898+0000","last_became_peered":"2023-10-08T09:53:56.896898+0000","last_unstale":"2023-10-08T09:55:46.321933+0000","last_undegraded":"2023-10-08T09:55:46.321933+0000","last_fullsized":"2023-10-08T09:55:46.321933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:18.850298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256320+0000","last_change":"2023-10-08T09:53:56.895889+0000","last_active":"2023-10-08T09:55:46.256320+0000","last_peered":"2023-10-08T09:55:46.256320+0000","last_clean":"2023-10-08T09:55:46.256320+0000","last_became_active":"2023-10-08T09:53:56.895553+0000","last_became_peered":"2023-10-08T09:53:56.895553+0000","last_unstale":"2023-10-08T09:55:46.256320+0000","last_undegraded":"2023-10-08T09:55:46.256320+0000","last_fullsized":"2023-10-08T09:55:46.256320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:25.564577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253841+0000","last_change":"2023-10-08T09:53:56.896993+0000","last_active":"2023-10-08T09:55:46.253841+0000","last_peered":"2023-10-08T09:55:46.253841+0000","last_clean":"2023-10-08T09:55:46.253841+0000","last_became_active":"2023-10-08T09:53:56.896669+0000","last_became_peered":"2023-10-08T09:53:56.896669+0000","last_unstale":"2023-10-08T09:55:46.253841+0000","last_undegraded":"2023-10-08T09:55:46.253841+0000","last_fullsized":"2023-10-08T09:55:46.253841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:08.468485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787058+0000","last_change":"2023-10-08T09:53:56.900063+0000","last_active":"2023-10-08T09:55:46.787058+0000","last_peered":"2023-10-08T09:55:46.787058+0000","last_clean":"2023-10-08T09:55:46.787058+0000","last_became_active":"2023-10-08T09:53:56.899766+0000","last_became_peered":"2023-10-08T09:53:56.899766+0000","last_unstale":"2023-10-08T09:55:46.787058+0000","last_undegraded":"2023-10-08T09:55:46.787058+0000","last_fullsized":"2023-10-08T09:55:46.787058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:21:11.855080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786778+0000","last_change":"2023-10-08T09:54:47.372615+0000","last_active":"2023-10-08T09:55:46.786778+0000","last_peered":"2023-10-08T09:55:46.786778+0000","last_clean":"2023-10-08T09:55:46.786778+0000","last_became_active":"2023-10-08T09:54:47.371960+0000","last_became_peered":"2023-10-08T09:54:47.371960+0000","last_unstale":"2023-10-08T09:55:46.786778+0000","last_undegraded":"2023-10-08T09:55:46.786778+0000","last_fullsized":"2023-10-08T09:55:46.786778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:45.836464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253805+0000","last_change":"2023-10-08T09:53:56.205317+0000","last_active":"2023-10-08T09:55:46.253805+0000","last_peered":"2023-10-08T09:55:46.253805+0000","last_clean":"2023-10-08T09:55:46.253805+0000","last_became_active":"2023-10-08T09:53:56.204982+0000","last_became_peered":"2023-10-08T09:53:56.204982+0000","last_unstale":"2023-10-08T09:55:46.253805+0000","last_undegraded":"2023-10-08T09:55:46.253805+0000","last_fullsized":"2023-10-08T09:55:46.253805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.090285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321977+0000","last_change":"2023-10-08T09:53:56.900953+0000","last_active":"2023-10-08T09:55:46.321977+0000","last_peered":"2023-10-08T09:55:46.321977+0000","last_clean":"2023-10-08T09:55:46.321977+0000","last_became_active":"2023-10-08T09:53:56.900624+0000","last_became_peered":"2023-10-08T09:53:56.900624+0000","last_unstale":"2023-10-08T09:55:46.321977+0000","last_undegraded":"2023-10-08T09:55:46.321977+0000","last_fullsized":"2023-10-08T09:55:46.321977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:12.985046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317139+0000","last_change":"2023-10-08T09:54:01.250949+0000","last_active":"2023-10-08T09:55:46.317139+0000","last_peered":"2023-10-08T09:55:46.317139+0000","last_clean":"2023-10-08T09:55:46.317139+0000","last_became_active":"2023-10-08T09:54:01.250580+0000","last_became_peered":"2023-10-08T09:54:01.250580+0000","last_unstale":"2023-10-08T09:55:46.317139+0000","last_undegraded":"2023-10-08T09:55:46.317139+0000","last_fullsized":"2023-10-08T09:55:46.317139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:35.845293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322041+0000","last_change":"2023-10-08T09:53:56.206825+0000","last_active":"2023-10-08T09:55:46.322041+0000","last_peered":"2023-10-08T09:55:46.322041+0000","last_clean":"2023-10-08T09:55:46.322041+0000","last_became_active":"2023-10-08T09:53:56.206541+0000","last_became_peered":"2023-10-08T09:53:56.206541+0000","last_unstale":"2023-10-08T09:55:46.322041+0000","last_undegraded":"2023-10-08T09:55:46.322041+0000","last_fullsized":"2023-10-08T09:55:46.322041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:42.754444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.146053+0000","last_change":"2023-10-08T09:53:56.902274+0000","last_active":"2023-10-08T09:55:47.146053+0000","last_peered":"2023-10-08T09:55:47.146053+0000","last_clean":"2023-10-08T09:55:47.146053+0000","last_became_active":"2023-10-08T09:53:56.901982+0000","last_became_peered":"2023-10-08T09:53:56.901982+0000","last_unstale":"2023-10-08T09:55:47.146053+0000","last_undegraded":"2023-10-08T09:55:47.146053+0000","last_fullsized":"2023-10-08T09:55:47.146053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:02.995784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317320+0000","last_change":"2023-10-08T09:54:01.246170+0000","last_active":"2023-10-08T09:55:46.317320+0000","last_peered":"2023-10-08T09:55:46.317320+0000","last_clean":"2023-10-08T09:55:46.317320+0000","last_became_active":"2023-10-08T09:54:01.245461+0000","last_became_peered":"2023-10-08T09:54:01.245461+0000","last_unstale":"2023-10-08T09:55:46.317320+0000","last_undegraded":"2023-10-08T09:55:46.317320+0000","last_fullsized":"2023-10-08T09:55:46.317320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:14:38.747791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722326+0000","last_change":"2023-10-08T09:53:56.205886+0000","last_active":"2023-10-08T09:55:46.722326+0000","last_peered":"2023-10-08T09:55:46.722326+0000","last_clean":"2023-10-08T09:55:46.722326+0000","last_became_active":"2023-10-08T09:53:56.205577+0000","last_became_peered":"2023-10-08T09:53:56.205577+0000","last_unstale":"2023-10-08T09:55:46.722326+0000","last_undegraded":"2023-10-08T09:55:46.722326+0000","last_fullsized":"2023-10-08T09:55:46.722326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:52.454450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322332+0000","last_change":"2023-10-08T09:53:56.891307+0000","last_active":"2023-10-08T09:55:46.322332+0000","last_peered":"2023-10-08T09:55:46.322332+0000","last_clean":"2023-10-08T09:55:46.322332+0000","last_became_active":"2023-10-08T09:53:56.891003+0000","last_became_peered":"2023-10-08T09:53:56.891003+0000","last_unstale":"2023-10-08T09:55:46.322332+0000","last_undegraded":"2023-10-08T09:55:46.322332+0000","last_fullsized":"2023-10-08T09:55:46.322332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:02.357358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317255+0000","last_change":"2023-10-08T09:54:01.254854+0000","last_active":"2023-10-08T09:55:46.317255+0000","last_peered":"2023-10-08T09:55:46.317255+0000","last_clean":"2023-10-08T09:55:46.317255+0000","last_became_active":"2023-10-08T09:54:01.254333+0000","last_became_peered":"2023-10-08T09:54:01.254333+0000","last_unstale":"2023-10-08T09:55:46.317255+0000","last_undegraded":"2023-10-08T09:55:46.317255+0000","last_fullsized":"2023-10-08T09:55:46.317255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:36.878198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722320+0000","last_change":"2023-10-08T09:53:56.201668+0000","last_active":"2023-10-08T09:55:46.722320+0000","last_peered":"2023-10-08T09:55:46.722320+0000","last_clean":"2023-10-08T09:55:46.722320+0000","last_became_active":"2023-10-08T09:53:56.201395+0000","last_became_peered":"2023-10-08T09:53:56.201395+0000","last_unstale":"2023-10-08T09:55:46.722320+0000","last_undegraded":"2023-10-08T09:55:46.722320+0000","last_fullsized":"2023-10-08T09:55:46.722320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:49.956179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319415+0000","last_change":"2023-10-08T09:53:56.894623+0000","last_active":"2023-10-08T09:55:46.319415+0000","last_peered":"2023-10-08T09:55:46.319415+0000","last_clean":"2023-10-08T09:55:46.319415+0000","last_became_active":"2023-10-08T09:53:56.894322+0000","last_became_peered":"2023-10-08T09:53:56.894322+0000","last_unstale":"2023-10-08T09:55:46.319415+0000","last_undegraded":"2023-10-08T09:55:46.319415+0000","last_fullsized":"2023-10-08T09:55:46.319415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:56.792269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786977+0000","last_change":"2023-10-08T09:54:01.242264+0000","last_active":"2023-10-08T09:55:46.786977+0000","last_peered":"2023-10-08T09:55:46.786977+0000","last_clean":"2023-10-08T09:55:46.786977+0000","last_became_active":"2023-10-08T09:54:01.241907+0000","last_became_peered":"2023-10-08T09:54:01.241907+0000","last_unstale":"2023-10-08T09:55:46.786977+0000","last_undegraded":"2023-10-08T09:55:46.786977+0000","last_fullsized":"2023-10-08T09:55:46.786977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:14.899094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:48.325098+0000","last_change":"2023-10-08T09:53:56.205967+0000","last_active":"2023-10-08T09:55:48.325098+0000","last_peered":"2023-10-08T09:55:48.325098+0000","last_clean":"2023-10-08T09:55:48.325098+0000","last_became_active":"2023-10-08T09:53:56.205650+0000","last_became_peered":"2023-10-08T09:53:56.205650+0000","last_unstale":"2023-10-08T09:55:48.325098+0000","last_undegraded":"2023-10-08T09:55:48.325098+0000","last_fullsized":"2023-10-08T09:55:48.325098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:40.077577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722196+0000","last_change":"2023-10-08T09:53:56.889209+0000","last_active":"2023-10-08T09:55:46.722196+0000","last_peered":"2023-10-08T09:55:46.722196+0000","last_clean":"2023-10-08T09:55:46.722196+0000","last_became_active":"2023-10-08T09:53:56.888921+0000","last_became_peered":"2023-10-08T09:53:56.888921+0000","last_unstale":"2023-10-08T09:55:46.722196+0000","last_undegraded":"2023-10-08T09:55:46.722196+0000","last_fullsized":"2023-10-08T09:55:46.722196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:42.200197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786805+0000","last_change":"2023-10-08T09:54:01.251190+0000","last_active":"2023-10-08T09:55:46.786805+0000","last_peered":"2023-10-08T09:55:46.786805+0000","last_clean":"2023-10-08T09:55:46.786805+0000","last_became_active":"2023-10-08T09:54:01.250802+0000","last_became_peered":"2023-10-08T09:54:01.250802+0000","last_unstale":"2023-10-08T09:55:46.786805+0000","last_undegraded":"2023-10-08T09:55:46.786805+0000","last_fullsized":"2023-10-08T09:55:46.786805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:16.945016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316010+0000","last_change":"2023-10-08T09:53:56.205102+0000","last_active":"2023-10-08T09:55:46.316010+0000","last_peered":"2023-10-08T09:55:46.316010+0000","last_clean":"2023-10-08T09:55:46.316010+0000","last_became_active":"2023-10-08T09:53:56.204797+0000","last_became_peered":"2023-10-08T09:53:56.204797+0000","last_unstale":"2023-10-08T09:55:46.316010+0000","last_undegraded":"2023-10-08T09:55:46.316010+0000","last_fullsized":"2023-10-08T09:55:46.316010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:53.979119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319584+0000","last_change":"2023-10-08T09:53:56.889552+0000","last_active":"2023-10-08T09:55:46.319584+0000","last_peered":"2023-10-08T09:55:46.319584+0000","last_clean":"2023-10-08T09:55:46.319584+0000","last_became_active":"2023-10-08T09:53:56.889261+0000","last_became_peered":"2023-10-08T09:53:56.889261+0000","last_unstale":"2023-10-08T09:55:46.319584+0000","last_undegraded":"2023-10-08T09:55:46.319584+0000","last_fullsized":"2023-10-08T09:55:46.319584+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.783589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256222+0000","last_change":"2023-10-08T09:54:01.245897+0000","last_active":"2023-10-08T09:55:46.256222+0000","last_peered":"2023-10-08T09:55:46.256222+0000","last_clean":"2023-10-08T09:55:46.256222+0000","last_became_active":"2023-10-08T09:54:01.245426+0000","last_became_peered":"2023-10-08T09:54:01.245426+0000","last_unstale":"2023-10-08T09:55:46.256222+0000","last_undegraded":"2023-10-08T09:55:46.256222+0000","last_fullsized":"2023-10-08T09:55:46.256222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:35.048197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314607+0000","last_change":"2023-10-08T09:53:56.197566+0000","last_active":"2023-10-08T09:55:46.314607+0000","last_peered":"2023-10-08T09:55:46.314607+0000","last_clean":"2023-10-08T09:55:46.314607+0000","last_became_active":"2023-10-08T09:53:56.197249+0000","last_became_peered":"2023-10-08T09:53:56.197249+0000","last_unstale":"2023-10-08T09:55:46.314607+0000","last_undegraded":"2023-10-08T09:55:46.314607+0000","last_fullsized":"2023-10-08T09:55:46.314607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:02.073075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321353+0000","last_change":"2023-10-08T09:53:56.884683+0000","last_active":"2023-10-08T09:55:46.321353+0000","last_peered":"2023-10-08T09:55:46.321353+0000","last_clean":"2023-10-08T09:55:46.321353+0000","last_became_active":"2023-10-08T09:53:56.884362+0000","last_became_peered":"2023-10-08T09:53:56.884362+0000","last_unstale":"2023-10-08T09:55:46.321353+0000","last_undegraded":"2023-10-08T09:55:46.321353+0000","last_fullsized":"2023-10-08T09:55:46.321353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:58.913716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145844+0000","last_change":"2023-10-08T09:54:01.255096+0000","last_active":"2023-10-08T09:55:47.145844+0000","last_peered":"2023-10-08T09:55:47.145844+0000","last_clean":"2023-10-08T09:55:47.145844+0000","last_became_active":"2023-10-08T09:54:01.254589+0000","last_became_peered":"2023-10-08T09:54:01.254589+0000","last_unstale":"2023-10-08T09:55:47.145844+0000","last_undegraded":"2023-10-08T09:55:47.145844+0000","last_fullsized":"2023-10-08T09:55:47.145844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:24.280743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318886+0000","last_change":"2023-10-08T09:53:56.206942+0000","last_active":"2023-10-08T09:55:46.318886+0000","last_peered":"2023-10-08T09:55:46.318886+0000","last_clean":"2023-10-08T09:55:46.318886+0000","last_became_active":"2023-10-08T09:53:56.206445+0000","last_became_peered":"2023-10-08T09:53:56.206445+0000","last_unstale":"2023-10-08T09:55:46.318886+0000","last_undegraded":"2023-10-08T09:55:46.318886+0000","last_fullsized":"2023-10-08T09:55:46.318886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:35.305354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253303+0000","last_change":"2023-10-08T09:53:56.895079+0000","last_active":"2023-10-08T09:55:46.253303+0000","last_peered":"2023-10-08T09:55:46.253303+0000","last_clean":"2023-10-08T09:55:46.253303+0000","last_became_active":"2023-10-08T09:53:56.894646+0000","last_became_peered":"2023-10-08T09:53:56.894646+0000","last_unstale":"2023-10-08T09:55:46.253303+0000","last_undegraded":"2023-10-08T09:55:46.253303+0000","last_fullsized":"2023-10-08T09:55:46.253303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:03.434226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321535+0000","last_change":"2023-10-08T09:54:01.243737+0000","last_active":"2023-10-08T09:55:46.321535+0000","last_peered":"2023-10-08T09:55:46.321535+0000","last_clean":"2023-10-08T09:55:46.321535+0000","last_became_active":"2023-10-08T09:54:01.243421+0000","last_became_peered":"2023-10-08T09:54:01.243421+0000","last_unstale":"2023-10-08T09:55:46.321535+0000","last_undegraded":"2023-10-08T09:55:46.321535+0000","last_fullsized":"2023-10-08T09:55:46.321535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:22.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:49.325875+0000","last_change":"2023-10-08T09:53:56.203823+0000","last_active":"2023-10-08T09:55:49.325875+0000","last_peered":"2023-10-08T09:55:49.325875+0000","last_clean":"2023-10-08T09:55:49.325875+0000","last_became_active":"2023-10-08T09:53:56.203505+0000","last_became_peered":"2023-10-08T09:53:56.203505+0000","last_unstale":"2023-10-08T09:55:49.325875+0000","last_undegraded":"2023-10-08T09:55:49.325875+0000","last_fullsized":"2023-10-08T09:55:49.325875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:22.476041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322213+0000","last_change":"2023-10-08T09:53:56.898083+0000","last_active":"2023-10-08T09:55:46.322213+0000","last_peered":"2023-10-08T09:55:46.322213+0000","last_clean":"2023-10-08T09:55:46.322213+0000","last_became_active":"2023-10-08T09:53:56.897804+0000","last_became_peered":"2023-10-08T09:53:56.897804+0000","last_unstale":"2023-10-08T09:55:46.322213+0000","last_undegraded":"2023-10-08T09:55:46.322213+0000","last_fullsized":"2023-10-08T09:55:46.322213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:12.187619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253147+0000","last_change":"2023-10-08T09:54:01.245330+0000","last_active":"2023-10-08T09:55:46.253147+0000","last_peered":"2023-10-08T09:55:46.253147+0000","last_clean":"2023-10-08T09:55:46.253147+0000","last_became_active":"2023-10-08T09:54:01.244921+0000","last_became_peered":"2023-10-08T09:54:01.244921+0000","last_unstale":"2023-10-08T09:55:46.253147+0000","last_undegraded":"2023-10-08T09:55:46.253147+0000","last_fullsized":"2023-10-08T09:55:46.253147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:14.885332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254260+0000","last_change":"2023-10-08T09:53:56.200642+0000","last_active":"2023-10-08T09:55:46.254260+0000","last_peered":"2023-10-08T09:55:46.254260+0000","last_clean":"2023-10-08T09:55:46.254260+0000","last_became_active":"2023-10-08T09:53:56.200343+0000","last_became_peered":"2023-10-08T09:53:56.200343+0000","last_unstale":"2023-10-08T09:55:46.254260+0000","last_undegraded":"2023-10-08T09:55:46.254260+0000","last_fullsized":"2023-10-08T09:55:46.254260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:01.239700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145955+0000","last_change":"2023-10-08T09:53:56.888484+0000","last_active":"2023-10-08T09:55:47.145955+0000","last_peered":"2023-10-08T09:55:47.145955+0000","last_clean":"2023-10-08T09:55:47.145955+0000","last_became_active":"2023-10-08T09:53:56.888163+0000","last_became_peered":"2023-10-08T09:53:56.888163+0000","last_unstale":"2023-10-08T09:55:47.145955+0000","last_undegraded":"2023-10-08T09:55:47.145955+0000","last_fullsized":"2023-10-08T09:55:47.145955+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:08.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317450+0000","last_change":"2023-10-08T09:54:01.246349+0000","last_active":"2023-10-08T09:55:46.317450+0000","last_peered":"2023-10-08T09:55:46.317450+0000","last_clean":"2023-10-08T09:55:46.317450+0000","last_became_active":"2023-10-08T09:54:01.245484+0000","last_became_peered":"2023-10-08T09:54:01.245484+0000","last_unstale":"2023-10-08T09:55:46.317450+0000","last_undegraded":"2023-10-08T09:55:46.317450+0000","last_fullsized":"2023-10-08T09:55:46.317450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:46.820231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319248+0000","last_change":"2023-10-08T09:53:56.201780+0000","last_active":"2023-10-08T09:55:46.319248+0000","last_peered":"2023-10-08T09:55:46.319248+0000","last_clean":"2023-10-08T09:55:46.319248+0000","last_became_active":"2023-10-08T09:53:56.201618+0000","last_became_peered":"2023-10-08T09:53:56.201618+0000","last_unstale":"2023-10-08T09:55:46.319248+0000","last_undegraded":"2023-10-08T09:55:46.319248+0000","last_fullsized":"2023-10-08T09:55:46.319248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:12.998310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322364+0000","last_change":"2023-10-08T09:53:56.884405+0000","last_active":"2023-10-08T09:55:46.322364+0000","last_peered":"2023-10-08T09:55:46.322364+0000","last_clean":"2023-10-08T09:55:46.322364+0000","last_became_active":"2023-10-08T09:53:56.884077+0000","last_became_peered":"2023-10-08T09:53:56.884077+0000","last_unstale":"2023-10-08T09:55:46.322364+0000","last_undegraded":"2023-10-08T09:55:46.322364+0000","last_fullsized":"2023-10-08T09:55:46.322364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:23:35.427316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253282+0000","last_change":"2023-10-08T09:54:01.241880+0000","last_active":"2023-10-08T09:55:46.253282+0000","last_peered":"2023-10-08T09:55:46.253282+0000","last_clean":"2023-10-08T09:55:46.253282+0000","last_became_active":"2023-10-08T09:54:01.241431+0000","last_became_peered":"2023-10-08T09:54:01.241431+0000","last_unstale":"2023-10-08T09:55:46.253282+0000","last_undegraded":"2023-10-08T09:55:46.253282+0000","last_fullsized":"2023-10-08T09:55:46.253282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:45.807709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'14","reported_seq":232,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:50.338454+0000","last_change":"2023-10-08T09:54:47.251279+0000","last_active":"2023-10-08T09:55:50.338454+0000","last_peered":"2023-10-08T09:55:50.338454+0000","last_clean":"2023-10-08T09:55:50.338454+0000","last_became_active":"2023-10-08T09:54:47.243512+0000","last_became_peered":"2023-10-08T09:54:47.243512+0000","last_unstale":"2023-10-08T09:55:50.338454+0000","last_undegraded":"2023-10-08T09:55:50.338454+0000","last_fullsized":"2023-10-08T09:55:50.338454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:54.079225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4102,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":171,"num_read_kb":90,"num_write":15,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722197+0000","last_change":"2023-10-08T09:53:56.891340+0000","last_active":"2023-10-08T09:55:46.722197+0000","last_peered":"2023-10-08T09:55:46.722197+0000","last_clean":"2023-10-08T09:55:46.722197+0000","last_became_active":"2023-10-08T09:53:56.891055+0000","last_became_peered":"2023-10-08T09:53:56.891055+0000","last_unstale":"2023-10-08T09:55:46.722197+0000","last_undegraded":"2023-10-08T09:55:46.722197+0000","last_fullsized":"2023-10-08T09:55:46.722197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:37.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316087+0000","last_change":"2023-10-08T09:54:47.372063+0000","last_active":"2023-10-08T09:55:46.316087+0000","last_peered":"2023-10-08T09:55:46.316087+0000","last_clean":"2023-10-08T09:55:46.316087+0000","last_became_active":"2023-10-08T09:54:47.371726+0000","last_became_peered":"2023-10-08T09:54:47.371726+0000","last_unstale":"2023-10-08T09:55:46.316087+0000","last_undegraded":"2023-10-08T09:55:46.316087+0000","last_fullsized":"2023-10-08T09:55:46.316087+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:15.974675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786794+0000","last_change":"2023-10-08T09:53:56.210174+0000","last_active":"2023-10-08T09:55:46.786794+0000","last_peered":"2023-10-08T09:55:46.786794+0000","last_clean":"2023-10-08T09:55:46.786794+0000","last_became_active":"2023-10-08T09:53:56.209888+0000","last_became_peered":"2023-10-08T09:53:56.209888+0000","last_unstale":"2023-10-08T09:55:46.786794+0000","last_undegraded":"2023-10-08T09:55:46.786794+0000","last_fullsized":"2023-10-08T09:55:46.786794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:30.573988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314846+0000","last_change":"2023-10-08T09:53:56.892260+0000","last_active":"2023-10-08T09:55:46.314846+0000","last_peered":"2023-10-08T09:55:46.314846+0000","last_clean":"2023-10-08T09:55:46.314846+0000","last_became_active":"2023-10-08T09:53:56.891950+0000","last_became_peered":"2023-10-08T09:53:56.891950+0000","last_unstale":"2023-10-08T09:55:46.314846+0000","last_undegraded":"2023-10-08T09:55:46.314846+0000","last_fullsized":"2023-10-08T09:55:46.314846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:50.784847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255796+0000","last_change":"2023-10-08T09:54:01.254967+0000","last_active":"2023-10-08T09:55:46.255796+0000","last_peered":"2023-10-08T09:55:46.255796+0000","last_clean":"2023-10-08T09:55:46.255796+0000","last_became_active":"2023-10-08T09:54:01.254535+0000","last_became_peered":"2023-10-08T09:54:01.254535+0000","last_unstale":"2023-10-08T09:55:46.255796+0000","last_undegraded":"2023-10-08T09:55:46.255796+0000","last_fullsized":"2023-10-08T09:55:46.255796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:01.764772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319462+0000","last_change":"2023-10-08T09:53:56.207706+0000","last_active":"2023-10-08T09:55:46.319462+0000","last_peered":"2023-10-08T09:55:46.319462+0000","last_clean":"2023-10-08T09:55:46.319462+0000","last_became_active":"2023-10-08T09:53:56.207452+0000","last_became_peered":"2023-10-08T09:53:56.207452+0000","last_unstale":"2023-10-08T09:55:46.319462+0000","last_undegraded":"2023-10-08T09:55:46.319462+0000","last_fullsized":"2023-10-08T09:55:46.319462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:11.117973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254261+0000","last_change":"2023-10-08T09:53:56.891125+0000","last_active":"2023-10-08T09:55:46.254261+0000","last_peered":"2023-10-08T09:55:46.254261+0000","last_clean":"2023-10-08T09:55:46.254261+0000","last_became_active":"2023-10-08T09:53:56.890823+0000","last_became_peered":"2023-10-08T09:53:56.890823+0000","last_unstale":"2023-10-08T09:55:46.254261+0000","last_undegraded":"2023-10-08T09:55:46.254261+0000","last_fullsized":"2023-10-08T09:55:46.254261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:19.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253707+0000","last_change":"2023-10-08T09:54:01.247616+0000","last_active":"2023-10-08T09:55:46.253707+0000","last_peered":"2023-10-08T09:55:46.253707+0000","last_clean":"2023-10-08T09:55:46.253707+0000","last_became_active":"2023-10-08T09:54:01.245104+0000","last_became_peered":"2023-10-08T09:54:01.245104+0000","last_unstale":"2023-10-08T09:55:46.253707+0000","last_undegraded":"2023-10-08T09:55:46.253707+0000","last_fullsized":"2023-10-08T09:55:46.253707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:29:15.358065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":112,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:48.324136+0000","last_change":"2023-10-08T09:53:56.205266+0000","last_active":"2023-10-08T09:55:48.324136+0000","last_peered":"2023-10-08T09:55:48.324136+0000","last_clean":"2023-10-08T09:55:48.324136+0000","last_became_active":"2023-10-08T09:53:56.204971+0000","last_became_peered":"2023-10-08T09:53:56.204971+0000","last_unstale":"2023-10-08T09:55:48.324136+0000","last_undegraded":"2023-10-08T09:55:48.324136+0000","last_fullsized":"2023-10-08T09:55:48.324136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:11.488834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253853+0000","last_change":"2023-10-08T09:53:56.894784+0000","last_active":"2023-10-08T09:55:46.253853+0000","last_peered":"2023-10-08T09:55:46.253853+0000","last_clean":"2023-10-08T09:55:46.253853+0000","last_became_active":"2023-10-08T09:53:56.894493+0000","last_became_peered":"2023-10-08T09:53:56.894493+0000","last_unstale":"2023-10-08T09:55:46.253853+0000","last_undegraded":"2023-10-08T09:55:46.253853+0000","last_fullsized":"2023-10-08T09:55:46.253853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:59.786820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315051+0000","last_change":"2023-10-08T09:54:01.254716+0000","last_active":"2023-10-08T09:55:46.315051+0000","last_peered":"2023-10-08T09:55:46.315051+0000","last_clean":"2023-10-08T09:55:46.315051+0000","last_became_active":"2023-10-08T09:54:01.254243+0000","last_became_peered":"2023-10-08T09:54:01.254243+0000","last_unstale":"2023-10-08T09:55:46.315051+0000","last_undegraded":"2023-10-08T09:55:46.315051+0000","last_fullsized":"2023-10-08T09:55:46.315051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:26.656584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721858+0000","last_change":"2023-10-08T09:54:47.261776+0000","last_active":"2023-10-08T09:55:46.721858+0000","last_peered":"2023-10-08T09:55:46.721858+0000","last_clean":"2023-10-08T09:55:46.721858+0000","last_became_active":"2023-10-08T09:54:47.245120+0000","last_became_peered":"2023-10-08T09:54:47.245120+0000","last_unstale":"2023-10-08T09:55:46.721858+0000","last_undegraded":"2023-10-08T09:55:46.721858+0000","last_fullsized":"2023-10-08T09:55:46.721858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:27.257566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319008+0000","last_change":"2023-10-08T09:53:56.889137+0000","last_active":"2023-10-08T09:55:46.319008+0000","last_peered":"2023-10-08T09:55:46.319008+0000","last_clean":"2023-10-08T09:55:46.319008+0000","last_became_active":"2023-10-08T09:53:56.888783+0000","last_became_peered":"2023-10-08T09:53:56.888783+0000","last_unstale":"2023-10-08T09:55:46.319008+0000","last_undegraded":"2023-10-08T09:55:46.319008+0000","last_fullsized":"2023-10-08T09:55:46.319008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:41.221521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318949+0000","last_change":"2023-10-08T09:54:01.239101+0000","last_active":"2023-10-08T09:55:46.318949+0000","last_peered":"2023-10-08T09:55:46.318949+0000","last_clean":"2023-10-08T09:55:46.318949+0000","last_became_active":"2023-10-08T09:54:01.238723+0000","last_became_peered":"2023-10-08T09:54:01.238723+0000","last_unstale":"2023-10-08T09:55:46.318949+0000","last_undegraded":"2023-10-08T09:55:46.318949+0000","last_fullsized":"2023-10-08T09:55:46.318949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:00.159160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256224+0000","last_change":"2023-10-08T09:54:47.267960+0000","last_active":"2023-10-08T09:55:46.256224+0000","last_peered":"2023-10-08T09:55:46.256224+0000","last_clean":"2023-10-08T09:55:46.256224+0000","last_became_active":"2023-10-08T09:54:47.250045+0000","last_became_peered":"2023-10-08T09:54:47.250045+0000","last_unstale":"2023-10-08T09:55:46.256224+0000","last_undegraded":"2023-10-08T09:55:46.256224+0000","last_fullsized":"2023-10-08T09:55:46.256224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:17.473086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256112+0000","last_change":"2023-10-08T09:53:56.892732+0000","last_active":"2023-10-08T09:55:46.256112+0000","last_peered":"2023-10-08T09:55:46.256112+0000","last_clean":"2023-10-08T09:55:46.256112+0000","last_became_active":"2023-10-08T09:53:56.892438+0000","last_became_peered":"2023-10-08T09:53:56.892438+0000","last_unstale":"2023-10-08T09:55:46.256112+0000","last_undegraded":"2023-10-08T09:55:46.256112+0000","last_fullsized":"2023-10-08T09:55:46.256112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:39.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722119+0000","last_change":"2023-10-08T09:54:47.245612+0000","last_active":"2023-10-08T09:55:46.722119+0000","last_peered":"2023-10-08T09:55:46.722119+0000","last_clean":"2023-10-08T09:55:46.722119+0000","last_became_active":"2023-10-08T09:54:47.245175+0000","last_became_peered":"2023-10-08T09:54:47.245175+0000","last_unstale":"2023-10-08T09:55:46.722119+0000","last_undegraded":"2023-10-08T09:55:46.722119+0000","last_fullsized":"2023-10-08T09:55:46.722119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:05.877929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253589+0000","last_change":"2023-10-08T09:53:56.208620+0000","last_active":"2023-10-08T09:55:46.253589+0000","last_peered":"2023-10-08T09:55:46.253589+0000","last_clean":"2023-10-08T09:55:46.253589+0000","last_became_active":"2023-10-08T09:53:56.208299+0000","last_became_peered":"2023-10-08T09:53:56.208299+0000","last_unstale":"2023-10-08T09:55:46.253589+0000","last_undegraded":"2023-10-08T09:55:46.253589+0000","last_fullsized":"2023-10-08T09:55:46.253589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:08.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316199+0000","last_change":"2023-10-08T09:53:56.890649+0000","last_active":"2023-10-08T09:55:46.316199+0000","last_peered":"2023-10-08T09:55:46.316199+0000","last_clean":"2023-10-08T09:55:46.316199+0000","last_became_active":"2023-10-08T09:53:56.890346+0000","last_became_peered":"2023-10-08T09:53:56.890346+0000","last_unstale":"2023-10-08T09:55:46.316199+0000","last_undegraded":"2023-10-08T09:55:46.316199+0000","last_fullsized":"2023-10-08T09:55:46.316199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:20.345079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314879+0000","last_change":"2023-10-08T09:54:01.252001+0000","last_active":"2023-10-08T09:55:46.314879+0000","last_peered":"2023-10-08T09:55:46.314879+0000","last_clean":"2023-10-08T09:55:46.314879+0000","last_became_active":"2023-10-08T09:54:01.251603+0000","last_became_peered":"2023-10-08T09:54:01.251603+0000","last_unstale":"2023-10-08T09:55:46.314879+0000","last_undegraded":"2023-10-08T09:55:46.314879+0000","last_fullsized":"2023-10-08T09:55:46.314879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:40.911153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254174+0000","last_change":"2023-10-08T09:53:56.203015+0000","last_active":"2023-10-08T09:55:46.254174+0000","last_peered":"2023-10-08T09:55:46.254174+0000","last_clean":"2023-10-08T09:55:46.254174+0000","last_became_active":"2023-10-08T09:53:56.202703+0000","last_became_peered":"2023-10-08T09:53:56.202703+0000","last_unstale":"2023-10-08T09:55:46.254174+0000","last_undegraded":"2023-10-08T09:55:46.254174+0000","last_fullsized":"2023-10-08T09:55:46.254174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:16.971027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.316423+0000","last_change":"2023-10-08T09:53:56.892195+0000","last_active":"2023-10-08T09:55:46.316423+0000","last_peered":"2023-10-08T09:55:46.316423+0000","last_clean":"2023-10-08T09:55:46.316423+0000","last_became_active":"2023-10-08T09:53:56.891956+0000","last_became_peered":"2023-10-08T09:53:56.891956+0000","last_unstale":"2023-10-08T09:55:46.316423+0000","last_undegraded":"2023-10-08T09:55:46.316423+0000","last_fullsized":"2023-10-08T09:55:46.316423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:05.218351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314834+0000","last_change":"2023-10-08T09:54:01.303687+0000","last_active":"2023-10-08T09:55:46.314834+0000","last_peered":"2023-10-08T09:55:46.314834+0000","last_clean":"2023-10-08T09:55:46.314834+0000","last_became_active":"2023-10-08T09:54:01.303269+0000","last_became_peered":"2023-10-08T09:54:01.303269+0000","last_unstale":"2023-10-08T09:55:46.314834+0000","last_undegraded":"2023-10-08T09:55:46.314834+0000","last_fullsized":"2023-10-08T09:55:46.314834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:09.563395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319178+0000","last_change":"2023-10-08T09:53:56.897894+0000","last_active":"2023-10-08T09:55:46.319178+0000","last_peered":"2023-10-08T09:55:46.319178+0000","last_clean":"2023-10-08T09:55:46.319178+0000","last_became_active":"2023-10-08T09:53:56.897585+0000","last_became_peered":"2023-10-08T09:53:56.897585+0000","last_unstale":"2023-10-08T09:55:46.319178+0000","last_undegraded":"2023-10-08T09:55:46.319178+0000","last_fullsized":"2023-10-08T09:55:46.319178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:56.011035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:49.324883+0000","last_change":"2023-10-08T09:53:56.207796+0000","last_active":"2023-10-08T09:55:49.324883+0000","last_peered":"2023-10-08T09:55:49.324883+0000","last_clean":"2023-10-08T09:55:49.324883+0000","last_became_active":"2023-10-08T09:53:56.207541+0000","last_became_peered":"2023-10-08T09:53:56.207541+0000","last_unstale":"2023-10-08T09:55:49.324883+0000","last_undegraded":"2023-10-08T09:55:49.324883+0000","last_fullsized":"2023-10-08T09:55:49.324883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:15.832690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254261+0000","last_change":"2023-10-08T09:54:01.241114+0000","last_active":"2023-10-08T09:55:46.254261+0000","last_peered":"2023-10-08T09:55:46.254261+0000","last_clean":"2023-10-08T09:55:46.254261+0000","last_became_active":"2023-10-08T09:54:01.240695+0000","last_became_peered":"2023-10-08T09:54:01.240695+0000","last_unstale":"2023-10-08T09:55:46.254261+0000","last_undegraded":"2023-10-08T09:55:46.254261+0000","last_fullsized":"2023-10-08T09:55:46.254261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:27.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.359149+0000","last_change":"2023-10-08T09:55:47.359149+0000","last_active":"2023-10-08T09:55:47.359149+0000","last_peered":"2023-10-08T09:55:47.359149+0000","last_clean":"2023-10-08T09:55:47.359149+0000","last_became_active":"2023-10-08T09:55:47.358812+0000","last_became_peered":"2023-10-08T09:55:47.358812+0000","last_unstale":"2023-10-08T09:55:47.359149+0000","last_undegraded":"2023-10-08T09:55:47.359149+0000","last_fullsized":"2023-10-08T09:55:47.359149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:21.157268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317403+0000","last_change":"2023-10-08T09:53:56.197625+0000","last_active":"2023-10-08T09:55:46.317403+0000","last_peered":"2023-10-08T09:55:46.317403+0000","last_clean":"2023-10-08T09:55:46.317403+0000","last_became_active":"2023-10-08T09:53:56.197300+0000","last_became_peered":"2023-10-08T09:53:56.197300+0000","last_unstale":"2023-10-08T09:55:46.317403+0000","last_undegraded":"2023-10-08T09:55:46.317403+0000","last_fullsized":"2023-10-08T09:55:46.317403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:51.963778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253732+0000","last_change":"2023-10-08T09:52:59.137242+0000","last_active":"2023-10-08T09:55:46.253732+0000","last_peered":"2023-10-08T09:55:46.253732+0000","last_clean":"2023-10-08T09:55:46.253732+0000","last_became_active":"2023-10-08T09:52:59.136715+0000","last_became_peered":"2023-10-08T09:52:59.136715+0000","last_unstale":"2023-10-08T09:55:46.253732+0000","last_undegraded":"2023-10-08T09:55:46.253732+0000","last_fullsized":"2023-10-08T09:55:46.253732+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:50:14.128871+0000","last_clean_scrub_stamp":"2023-10-08T09:50:14.128871+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:31.434685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319385+0000","last_change":"2023-10-08T09:54:01.304529+0000","last_active":"2023-10-08T09:55:46.319385+0000","last_peered":"2023-10-08T09:55:46.319385+0000","last_clean":"2023-10-08T09:55:46.319385+0000","last_became_active":"2023-10-08T09:54:01.304133+0000","last_became_peered":"2023-10-08T09:54:01.304133+0000","last_unstale":"2023-10-08T09:55:46.319385+0000","last_undegraded":"2023-10-08T09:55:46.319385+0000","last_fullsized":"2023-10-08T09:55:46.319385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:24.710937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314516+0000","last_change":"2023-10-08T09:54:47.245430+0000","last_active":"2023-10-08T09:55:46.314516+0000","last_peered":"2023-10-08T09:55:46.314516+0000","last_clean":"2023-10-08T09:55:46.314516+0000","last_became_active":"2023-10-08T09:54:47.245076+0000","last_became_peered":"2023-10-08T09:54:47.245076+0000","last_unstale":"2023-10-08T09:55:46.314516+0000","last_undegraded":"2023-10-08T09:55:46.314516+0000","last_fullsized":"2023-10-08T09:55:46.314516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:43.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253790+0000","last_change":"2023-10-08T09:53:56.202930+0000","last_active":"2023-10-08T09:55:46.253790+0000","last_peered":"2023-10-08T09:55:46.253790+0000","last_clean":"2023-10-08T09:55:46.253790+0000","last_became_active":"2023-10-08T09:53:56.202619+0000","last_became_peered":"2023-10-08T09:53:56.202619+0000","last_unstale":"2023-10-08T09:55:46.253790+0000","last_undegraded":"2023-10-08T09:55:46.253790+0000","last_fullsized":"2023-10-08T09:55:46.253790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:51.093985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255774+0000","last_change":"2023-10-08T09:54:01.251608+0000","last_active":"2023-10-08T09:55:46.255774+0000","last_peered":"2023-10-08T09:55:46.255774+0000","last_clean":"2023-10-08T09:55:46.255774+0000","last_became_active":"2023-10-08T09:54:01.251215+0000","last_became_peered":"2023-10-08T09:54:01.251215+0000","last_unstale":"2023-10-08T09:55:46.255774+0000","last_undegraded":"2023-10-08T09:55:46.255774+0000","last_fullsized":"2023-10-08T09:55:46.255774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:25.445390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255918+0000","last_change":"2023-10-08T09:54:47.264240+0000","last_active":"2023-10-08T09:55:46.255918+0000","last_peered":"2023-10-08T09:55:46.255918+0000","last_clean":"2023-10-08T09:55:46.255918+0000","last_became_active":"2023-10-08T09:54:47.245582+0000","last_became_peered":"2023-10-08T09:54:47.245582+0000","last_unstale":"2023-10-08T09:55:46.255918+0000","last_undegraded":"2023-10-08T09:55:46.255918+0000","last_fullsized":"2023-10-08T09:55:46.255918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:14.908777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314966+0000","last_change":"2023-10-08T09:53:56.901706+0000","last_active":"2023-10-08T09:55:46.314966+0000","last_peered":"2023-10-08T09:55:46.314966+0000","last_clean":"2023-10-08T09:55:46.314966+0000","last_became_active":"2023-10-08T09:53:56.901404+0000","last_became_peered":"2023-10-08T09:53:56.901404+0000","last_unstale":"2023-10-08T09:55:46.314966+0000","last_undegraded":"2023-10-08T09:55:46.314966+0000","last_fullsized":"2023-10-08T09:55:46.314966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:40.818586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321320+0000","last_change":"2023-10-08T09:54:01.304000+0000","last_active":"2023-10-08T09:55:46.321320+0000","last_peered":"2023-10-08T09:55:46.321320+0000","last_clean":"2023-10-08T09:55:46.321320+0000","last_became_active":"2023-10-08T09:54:01.303654+0000","last_became_peered":"2023-10-08T09:54:01.303654+0000","last_unstale":"2023-10-08T09:55:46.321320+0000","last_undegraded":"2023-10-08T09:55:46.321320+0000","last_fullsized":"2023-10-08T09:55:46.321320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.891903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.253843+0000","last_change":"2023-10-08T09:53:56.211564+0000","last_active":"2023-10-08T09:55:46.253843+0000","last_peered":"2023-10-08T09:55:46.253843+0000","last_clean":"2023-10-08T09:55:46.253843+0000","last_became_active":"2023-10-08T09:53:56.211227+0000","last_became_peered":"2023-10-08T09:53:56.211227+0000","last_unstale":"2023-10-08T09:55:46.253843+0000","last_undegraded":"2023-10-08T09:55:46.253843+0000","last_fullsized":"2023-10-08T09:55:46.253843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:41.553750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321884+0000","last_change":"2023-10-08T09:53:56.890977+0000","last_active":"2023-10-08T09:55:46.321884+0000","last_peered":"2023-10-08T09:55:46.321884+0000","last_clean":"2023-10-08T09:55:46.321884+0000","last_became_active":"2023-10-08T09:53:56.890710+0000","last_became_peered":"2023-10-08T09:53:56.890710+0000","last_unstale":"2023-10-08T09:55:46.321884+0000","last_undegraded":"2023-10-08T09:55:46.321884+0000","last_fullsized":"2023-10-08T09:55:46.321884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:55.222809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786506+0000","last_change":"2023-10-08T09:54:01.241615+0000","last_active":"2023-10-08T09:55:46.786506+0000","last_peered":"2023-10-08T09:55:46.786506+0000","last_clean":"2023-10-08T09:55:46.786506+0000","last_became_active":"2023-10-08T09:54:01.241256+0000","last_became_peered":"2023-10-08T09:54:01.241256+0000","last_unstale":"2023-10-08T09:55:46.786506+0000","last_undegraded":"2023-10-08T09:55:46.786506+0000","last_fullsized":"2023-10-08T09:55:46.786506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:13.614989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317840+0000","last_change":"2023-10-08T09:53:56.196101+0000","last_active":"2023-10-08T09:55:46.317840+0000","last_peered":"2023-10-08T09:55:46.317840+0000","last_clean":"2023-10-08T09:55:46.317840+0000","last_became_active":"2023-10-08T09:53:56.195768+0000","last_became_peered":"2023-10-08T09:53:56.195768+0000","last_unstale":"2023-10-08T09:55:46.317840+0000","last_undegraded":"2023-10-08T09:55:46.317840+0000","last_fullsized":"2023-10-08T09:55:46.317840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:28.179515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145981+0000","last_change":"2023-10-08T09:53:56.891522+0000","last_active":"2023-10-08T09:55:47.145981+0000","last_peered":"2023-10-08T09:55:47.145981+0000","last_clean":"2023-10-08T09:55:47.145981+0000","last_became_active":"2023-10-08T09:53:56.891226+0000","last_became_peered":"2023-10-08T09:53:56.891226+0000","last_unstale":"2023-10-08T09:55:47.145981+0000","last_undegraded":"2023-10-08T09:55:47.145981+0000","last_fullsized":"2023-10-08T09:55:47.145981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:23.284917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317664+0000","last_change":"2023-10-08T09:54:01.304658+0000","last_active":"2023-10-08T09:55:46.317664+0000","last_peered":"2023-10-08T09:55:46.317664+0000","last_clean":"2023-10-08T09:55:46.317664+0000","last_became_active":"2023-10-08T09:54:01.304214+0000","last_became_peered":"2023-10-08T09:54:01.304214+0000","last_unstale":"2023-10-08T09:55:46.317664+0000","last_undegraded":"2023-10-08T09:55:46.317664+0000","last_fullsized":"2023-10-08T09:55:46.317664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:49.752075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321853+0000","last_change":"2023-10-08T09:53:56.200894+0000","last_active":"2023-10-08T09:55:46.321853+0000","last_peered":"2023-10-08T09:55:46.321853+0000","last_clean":"2023-10-08T09:55:46.321853+0000","last_became_active":"2023-10-08T09:53:56.200588+0000","last_became_peered":"2023-10-08T09:53:56.200588+0000","last_unstale":"2023-10-08T09:55:46.321853+0000","last_undegraded":"2023-10-08T09:55:46.321853+0000","last_fullsized":"2023-10-08T09:55:46.321853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:00.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.721854+0000","last_change":"2023-10-08T09:53:56.890081+0000","last_active":"2023-10-08T09:55:46.721854+0000","last_peered":"2023-10-08T09:55:46.721854+0000","last_clean":"2023-10-08T09:55:46.721854+0000","last_became_active":"2023-10-08T09:53:56.889792+0000","last_became_peered":"2023-10-08T09:53:56.889792+0000","last_unstale":"2023-10-08T09:55:46.721854+0000","last_undegraded":"2023-10-08T09:55:46.721854+0000","last_fullsized":"2023-10-08T09:55:46.721854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:24.964273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722292+0000","last_change":"2023-10-08T09:54:47.372412+0000","last_active":"2023-10-08T09:55:46.722292+0000","last_peered":"2023-10-08T09:55:46.722292+0000","last_clean":"2023-10-08T09:55:46.722292+0000","last_became_active":"2023-10-08T09:54:47.371964+0000","last_became_peered":"2023-10-08T09:54:47.371964+0000","last_unstale":"2023-10-08T09:55:46.722292+0000","last_undegraded":"2023-10-08T09:55:46.722292+0000","last_fullsized":"2023-10-08T09:55:46.722292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:20.975377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321433+0000","last_change":"2023-10-08T09:53:56.202741+0000","last_active":"2023-10-08T09:55:46.321433+0000","last_peered":"2023-10-08T09:55:46.321433+0000","last_clean":"2023-10-08T09:55:46.321433+0000","last_became_active":"2023-10-08T09:53:56.202449+0000","last_became_peered":"2023-10-08T09:53:56.202449+0000","last_unstale":"2023-10-08T09:55:46.321433+0000","last_undegraded":"2023-10-08T09:55:46.321433+0000","last_fullsized":"2023-10-08T09:55:46.321433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:29.044277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255634+0000","last_change":"2023-10-08T09:53:56.897814+0000","last_active":"2023-10-08T09:55:46.255634+0000","last_peered":"2023-10-08T09:55:46.255634+0000","last_clean":"2023-10-08T09:55:46.255634+0000","last_became_active":"2023-10-08T09:53:56.897498+0000","last_became_peered":"2023-10-08T09:53:56.897498+0000","last_unstale":"2023-10-08T09:55:46.255634+0000","last_undegraded":"2023-10-08T09:55:46.255634+0000","last_fullsized":"2023-10-08T09:55:46.255634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:20.578860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317684+0000","last_change":"2023-10-08T09:54:01.246035+0000","last_active":"2023-10-08T09:55:46.317684+0000","last_peered":"2023-10-08T09:55:46.317684+0000","last_clean":"2023-10-08T09:55:46.317684+0000","last_became_active":"2023-10-08T09:54:01.245447+0000","last_became_peered":"2023-10-08T09:54:01.245447+0000","last_unstale":"2023-10-08T09:55:46.317684+0000","last_undegraded":"2023-10-08T09:55:46.317684+0000","last_fullsized":"2023-10-08T09:55:46.317684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:07.852804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786643+0000","last_change":"2023-10-08T09:53:56.207885+0000","last_active":"2023-10-08T09:55:46.786643+0000","last_peered":"2023-10-08T09:55:46.786643+0000","last_clean":"2023-10-08T09:55:46.786643+0000","last_became_active":"2023-10-08T09:53:56.207466+0000","last_became_peered":"2023-10-08T09:53:56.207466+0000","last_unstale":"2023-10-08T09:55:46.786643+0000","last_undegraded":"2023-10-08T09:55:46.786643+0000","last_fullsized":"2023-10-08T09:55:46.786643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.812053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321742+0000","last_change":"2023-10-08T09:53:56.889926+0000","last_active":"2023-10-08T09:55:46.321742+0000","last_peered":"2023-10-08T09:55:46.321742+0000","last_clean":"2023-10-08T09:55:46.321742+0000","last_became_active":"2023-10-08T09:53:56.889606+0000","last_became_peered":"2023-10-08T09:53:56.889606+0000","last_unstale":"2023-10-08T09:55:46.321742+0000","last_undegraded":"2023-10-08T09:55:46.321742+0000","last_fullsized":"2023-10-08T09:55:46.321742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:23:15.771919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.317547+0000","last_change":"2023-10-08T09:54:01.303805+0000","last_active":"2023-10-08T09:55:46.317547+0000","last_peered":"2023-10-08T09:55:46.317547+0000","last_clean":"2023-10-08T09:55:46.317547+0000","last_became_active":"2023-10-08T09:54:01.303331+0000","last_became_peered":"2023-10-08T09:54:01.303331+0000","last_unstale":"2023-10-08T09:55:46.317547+0000","last_undegraded":"2023-10-08T09:55:46.317547+0000","last_fullsized":"2023-10-08T09:55:46.317547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:38.677130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.722000+0000","last_change":"2023-10-08T09:53:56.198250+0000","last_active":"2023-10-08T09:55:46.722000+0000","last_peered":"2023-10-08T09:55:46.722000+0000","last_clean":"2023-10-08T09:55:46.722000+0000","last_became_active":"2023-10-08T09:53:56.197922+0000","last_became_peered":"2023-10-08T09:53:56.197922+0000","last_unstale":"2023-10-08T09:55:46.722000+0000","last_undegraded":"2023-10-08T09:55:46.722000+0000","last_fullsized":"2023-10-08T09:55:46.722000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:19.464914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254319+0000","last_change":"2023-10-08T09:53:56.895636+0000","last_active":"2023-10-08T09:55:46.254319+0000","last_peered":"2023-10-08T09:55:46.254319+0000","last_clean":"2023-10-08T09:55:46.254319+0000","last_became_active":"2023-10-08T09:53:56.895278+0000","last_became_peered":"2023-10-08T09:53:56.895278+0000","last_unstale":"2023-10-08T09:55:46.254319+0000","last_undegraded":"2023-10-08T09:55:46.254319+0000","last_fullsized":"2023-10-08T09:55:46.254319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:54.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321896+0000","last_change":"2023-10-08T09:54:01.255260+0000","last_active":"2023-10-08T09:55:46.321896+0000","last_peered":"2023-10-08T09:55:46.321896+0000","last_clean":"2023-10-08T09:55:46.321896+0000","last_became_active":"2023-10-08T09:54:01.254770+0000","last_became_peered":"2023-10-08T09:54:01.254770+0000","last_unstale":"2023-10-08T09:55:46.321896+0000","last_undegraded":"2023-10-08T09:55:46.321896+0000","last_fullsized":"2023-10-08T09:55:46.321896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:39.377693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786917+0000","last_change":"2023-10-08T09:53:56.203977+0000","last_active":"2023-10-08T09:55:46.786917+0000","last_peered":"2023-10-08T09:55:46.786917+0000","last_clean":"2023-10-08T09:55:46.786917+0000","last_became_active":"2023-10-08T09:53:56.203659+0000","last_became_peered":"2023-10-08T09:53:56.203659+0000","last_unstale":"2023-10-08T09:55:46.786917+0000","last_undegraded":"2023-10-08T09:55:46.786917+0000","last_fullsized":"2023-10-08T09:55:46.786917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:15.828440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254333+0000","last_change":"2023-10-08T09:53:56.894310+0000","last_active":"2023-10-08T09:55:46.254333+0000","last_peered":"2023-10-08T09:55:46.254333+0000","last_clean":"2023-10-08T09:55:46.254333+0000","last_became_active":"2023-10-08T09:53:56.894014+0000","last_became_peered":"2023-10-08T09:53:56.894014+0000","last_unstale":"2023-10-08T09:55:46.254333+0000","last_undegraded":"2023-10-08T09:55:46.254333+0000","last_fullsized":"2023-10-08T09:55:46.254333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:06.412829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145671+0000","last_change":"2023-10-08T09:54:01.245812+0000","last_active":"2023-10-08T09:55:47.145671+0000","last_peered":"2023-10-08T09:55:47.145671+0000","last_clean":"2023-10-08T09:55:47.145671+0000","last_became_active":"2023-10-08T09:54:01.245425+0000","last_became_peered":"2023-10-08T09:54:01.245425+0000","last_unstale":"2023-10-08T09:55:47.145671+0000","last_undegraded":"2023-10-08T09:55:47.145671+0000","last_fullsized":"2023-10-08T09:55:47.145671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:04.705586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.319384+0000","last_change":"2023-10-08T09:53:56.205857+0000","last_active":"2023-10-08T09:55:46.319384+0000","last_peered":"2023-10-08T09:55:46.319384+0000","last_clean":"2023-10-08T09:55:46.319384+0000","last_became_active":"2023-10-08T09:53:56.205578+0000","last_became_peered":"2023-10-08T09:53:56.205578+0000","last_unstale":"2023-10-08T09:55:46.319384+0000","last_undegraded":"2023-10-08T09:55:46.319384+0000","last_fullsized":"2023-10-08T09:55:46.319384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:47.677294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254084+0000","last_change":"2023-10-08T09:53:56.892481+0000","last_active":"2023-10-08T09:55:46.254084+0000","last_peered":"2023-10-08T09:55:46.254084+0000","last_clean":"2023-10-08T09:55:46.254084+0000","last_became_active":"2023-10-08T09:53:56.891413+0000","last_became_peered":"2023-10-08T09:53:56.891413+0000","last_unstale":"2023-10-08T09:55:46.254084+0000","last_undegraded":"2023-10-08T09:55:46.254084+0000","last_fullsized":"2023-10-08T09:55:46.254084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:46.170163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318763+0000","last_change":"2023-10-08T09:54:01.239766+0000","last_active":"2023-10-08T09:55:46.318763+0000","last_peered":"2023-10-08T09:55:46.318763+0000","last_clean":"2023-10-08T09:55:46.318763+0000","last_became_active":"2023-10-08T09:54:01.239353+0000","last_became_peered":"2023-10-08T09:54:01.239353+0000","last_unstale":"2023-10-08T09:55:46.318763+0000","last_undegraded":"2023-10-08T09:55:46.318763+0000","last_fullsized":"2023-10-08T09:55:46.318763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:54:00.218791+0000","last_clean_scrub_stamp":"2023-10-08T09:54:00.218791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:23.331873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315024+0000","last_change":"2023-10-08T09:53:56.210200+0000","last_active":"2023-10-08T09:55:46.315024+0000","last_peered":"2023-10-08T09:55:46.315024+0000","last_clean":"2023-10-08T09:55:46.315024+0000","last_became_active":"2023-10-08T09:53:56.209951+0000","last_became_peered":"2023-10-08T09:53:56.209951+0000","last_unstale":"2023-10-08T09:55:46.315024+0000","last_undegraded":"2023-10-08T09:55:46.315024+0000","last_fullsized":"2023-10-08T09:55:46.315024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:55.177120+0000","last_clean_scrub_stamp":"2023-10-08T09:53:55.177120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:10.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.254297+0000","last_change":"2023-10-08T09:53:56.892735+0000","last_active":"2023-10-08T09:55:46.254297+0000","last_peered":"2023-10-08T09:55:46.254297+0000","last_clean":"2023-10-08T09:55:46.254297+0000","last_became_active":"2023-10-08T09:53:56.891427+0000","last_became_peered":"2023-10-08T09:53:56.891427+0000","last_unstale":"2023-10-08T09:55:46.254297+0000","last_undegraded":"2023-10-08T09:55:46.254297+0000","last_fullsized":"2023-10-08T09:55:46.254297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:34.411709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255663+0000","last_change":"2023-10-08T09:53:56.901566+0000","last_active":"2023-10-08T09:55:46.255663+0000","last_peered":"2023-10-08T09:55:46.255663+0000","last_clean":"2023-10-08T09:55:46.255663+0000","last_became_active":"2023-10-08T09:53:56.901229+0000","last_became_peered":"2023-10-08T09:53:56.901229+0000","last_unstale":"2023-10-08T09:55:46.255663+0000","last_undegraded":"2023-10-08T09:55:46.255663+0000","last_fullsized":"2023-10-08T09:55:46.255663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:21.833818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786636+0000","last_change":"2023-10-08T09:53:56.900022+0000","last_active":"2023-10-08T09:55:46.786636+0000","last_peered":"2023-10-08T09:55:46.786636+0000","last_clean":"2023-10-08T09:55:46.786636+0000","last_became_active":"2023-10-08T09:53:56.899734+0000","last_became_peered":"2023-10-08T09:53:56.899734+0000","last_unstale":"2023-10-08T09:55:46.786636+0000","last_undegraded":"2023-10-08T09:55:46.786636+0000","last_fullsized":"2023-10-08T09:55:46.786636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.786645+0000","last_change":"2023-10-08T09:53:56.900189+0000","last_active":"2023-10-08T09:55:46.786645+0000","last_peered":"2023-10-08T09:55:46.786645+0000","last_clean":"2023-10-08T09:55:46.786645+0000","last_became_active":"2023-10-08T09:53:56.899804+0000","last_became_peered":"2023-10-08T09:53:56.899804+0000","last_unstale":"2023-10-08T09:55:46.786645+0000","last_undegraded":"2023-10-08T09:55:46.786645+0000","last_fullsized":"2023-10-08T09:55:46.786645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:10.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:47.145836+0000","last_change":"2023-10-08T09:53:56.889594+0000","last_active":"2023-10-08T09:55:47.145836+0000","last_peered":"2023-10-08T09:55:47.145836+0000","last_clean":"2023-10-08T09:55:47.145836+0000","last_became_active":"2023-10-08T09:53:56.889293+0000","last_became_peered":"2023-10-08T09:53:56.889293+0000","last_unstale":"2023-10-08T09:55:47.145836+0000","last_undegraded":"2023-10-08T09:55:47.145836+0000","last_fullsized":"2023-10-08T09:55:47.145836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:58.074062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.314807+0000","last_change":"2023-10-08T09:53:56.891111+0000","last_active":"2023-10-08T09:55:46.314807+0000","last_peered":"2023-10-08T09:55:46.314807+0000","last_clean":"2023-10-08T09:55:46.314807+0000","last_became_active":"2023-10-08T09:53:56.890799+0000","last_became_peered":"2023-10-08T09:53:56.890799+0000","last_unstale":"2023-10-08T09:55:46.314807+0000","last_undegraded":"2023-10-08T09:55:46.314807+0000","last_fullsized":"2023-10-08T09:55:46.314807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:00:37.428443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.256045+0000","last_change":"2023-10-08T09:53:56.890569+0000","last_active":"2023-10-08T09:55:46.256045+0000","last_peered":"2023-10-08T09:55:46.256045+0000","last_clean":"2023-10-08T09:55:46.256045+0000","last_became_active":"2023-10-08T09:53:56.890236+0000","last_became_peered":"2023-10-08T09:53:56.890236+0000","last_unstale":"2023-10-08T09:55:46.256045+0000","last_undegraded":"2023-10-08T09:55:46.256045+0000","last_fullsized":"2023-10-08T09:55:46.256045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:56.616774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.787148+0000","last_change":"2023-10-08T09:53:56.901926+0000","last_active":"2023-10-08T09:55:46.787148+0000","last_peered":"2023-10-08T09:55:46.787148+0000","last_clean":"2023-10-08T09:55:46.787148+0000","last_became_active":"2023-10-08T09:53:56.901642+0000","last_became_peered":"2023-10-08T09:53:56.901642+0000","last_unstale":"2023-10-08T09:55:46.787148+0000","last_undegraded":"2023-10-08T09:55:46.787148+0000","last_fullsized":"2023-10-08T09:55:46.787148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:11.775679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321983+0000","last_change":"2023-10-08T09:53:56.897795+0000","last_active":"2023-10-08T09:55:46.321983+0000","last_peered":"2023-10-08T09:55:46.321983+0000","last_clean":"2023-10-08T09:55:46.321983+0000","last_became_active":"2023-10-08T09:53:56.897476+0000","last_became_peered":"2023-10-08T09:53:56.897476+0000","last_unstale":"2023-10-08T09:55:46.321983+0000","last_undegraded":"2023-10-08T09:55:46.321983+0000","last_fullsized":"2023-10-08T09:55:46.321983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.035792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.318849+0000","last_change":"2023-10-08T09:53:56.889299+0000","last_active":"2023-10-08T09:55:46.318849+0000","last_peered":"2023-10-08T09:55:46.318849+0000","last_clean":"2023-10-08T09:55:46.318849+0000","last_became_active":"2023-10-08T09:53:56.888912+0000","last_became_peered":"2023-10-08T09:53:56.888912+0000","last_unstale":"2023-10-08T09:55:46.318849+0000","last_undegraded":"2023-10-08T09:55:46.318849+0000","last_fullsized":"2023-10-08T09:55:46.318849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:44.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.321400+0000","last_change":"2023-10-08T09:53:56.901733+0000","last_active":"2023-10-08T09:55:46.321400+0000","last_peered":"2023-10-08T09:55:46.321400+0000","last_clean":"2023-10-08T09:55:46.321400+0000","last_became_active":"2023-10-08T09:53:56.901454+0000","last_became_peered":"2023-10-08T09:53:56.901454+0000","last_unstale":"2023-10-08T09:55:46.321400+0000","last_undegraded":"2023-10-08T09:55:46.321400+0000","last_fullsized":"2023-10-08T09:55:46.321400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:42.728858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.322460+0000","last_change":"2023-10-08T09:53:56.900057+0000","last_active":"2023-10-08T09:55:46.322460+0000","last_peered":"2023-10-08T09:55:46.322460+0000","last_clean":"2023-10-08T09:55:46.322460+0000","last_became_active":"2023-10-08T09:53:56.899564+0000","last_became_peered":"2023-10-08T09:53:56.899564+0000","last_unstale":"2023-10-08T09:55:46.322460+0000","last_undegraded":"2023-10-08T09:55:46.322460+0000","last_fullsized":"2023-10-08T09:55:46.322460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:49.760292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.315062+0000","last_change":"2023-10-08T09:53:56.892888+0000","last_active":"2023-10-08T09:55:46.315062+0000","last_peered":"2023-10-08T09:55:46.315062+0000","last_clean":"2023-10-08T09:55:46.315062+0000","last_became_active":"2023-10-08T09:53:56.892577+0000","last_became_peered":"2023-10-08T09:53:56.892577+0000","last_unstale":"2023-10-08T09:55:46.315062+0000","last_undegraded":"2023-10-08T09:55:46.315062+0000","last_fullsized":"2023-10-08T09:55:46.315062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:43.982185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:55:46.255827+0000","last_change":"2023-10-08T09:53:56.889041+0000","last_active":"2023-10-08T09:55:46.255827+0000","last_peered":"2023-10-08T09:55:46.255827+0000","last_clean":"2023-10-08T09:55:46.255827+0000","last_became_active":"2023-10-08T09:53:56.888732+0000","last_became_peered":"2023-10-08T09:53:56.888732+0000","last_unstale":"2023-10-08T09:55:46.255827+0000","last_undegraded":"2023-10-08T09:55:46.255827+0000","last_fullsized":"2023-10-08T09:55:46.255827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:53:56.178636+0000","last_clean_scrub_stamp":"2023-10-08T09:53:56.178636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:21.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":533,"num_read_kb":280,"num_write":131,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":3998,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":23136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":495165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.68500000000000005}]},{"osd":1,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.71899999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75,"5min":1.091,"15min":1.091},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.879,"5min":1.073,"15min":1.073},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.33800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.111,"15min":1.111},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68200000000000005,"15min":0.69199999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0009999999999999,"5min":1.643,"15min":1.643},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.99099999999999999,"5min":1.054,"15min":1.054},"last":0.86199999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72799999999999998,"5min":1,"15min":1},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0580000000000001,"5min":1.179,"15min":1.179},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88500000000000001,"5min":1.133,"15min":1.133},"last":0.92500000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.96399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.044,"5min":1.589,"15min":1.589},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.874,"15min":0.874},"min":{"1min":0.62,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.038,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0109999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92600000000000005,"5min":1.026,"15min":1.026},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.85599999999999998},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.071,"5min":1.536,"15min":1.536},"last":0.98099999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1090587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41199999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.501}]},{"osd":4,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82799999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44400000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78100000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.498,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:53:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.55400000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67000000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:54:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.70399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93899999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88500000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58499999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83399999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95499999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74099999999999999}]}]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":495357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":0.746,"15min":0.746},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":1.339,"15min":1.339},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58499999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.622,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70299999999999996,"5min":1.125,"15min":1.125},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46600000000000003,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55900000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55500000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.308}]},{"osd":4,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75700000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82499999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":1.171,"15min":1.171},"last":0.43099999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:53:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70899999999999996,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79500000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46300000000000002}]},{"osd":8,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69499999999999995,"15min":0.73099999999999998},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83699999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56299999999999994}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.77900000000000003,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.52700000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.62}]}]},{"osd":8,"up_from":56,"seq":240518168622,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":495233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.6200000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.624}]},{"osd":1,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.49199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97599999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.409,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.99199999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89700000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.423,"5min":2.298,"15min":2.298},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.58899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0920000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0149999999999999,"5min":2.323,"15min":2.323},"last":0.88400000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76100000000000001,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.76400000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.71199999999999997,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.59399999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.052}]},{"osd":11,"last update":"Sun Oct 8 09:54:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0229999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1086699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56200000000000006,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.378},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53800000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.75600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.31,"5min":0.309,"15min":0.309},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.505,"15min":0.505},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66100000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79700000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.69999999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:55:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79400000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.008,"5min":0.754,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.7610000000000001,"5min":7.7610000000000001,"15min":7.7610000000000001},"last":0.60499999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73699999999999999},"min":{"1min":0.57199999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.752},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.78700000000000003},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93100000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.628}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70699999999999996,"15min":0.70999999999999996},"min":{"1min":0.54700000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.53500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79900000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.74199999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":494629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.38300000000000001,"15min":0.38},"min":{"1min":0.309,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.502,"5min":0.625,"15min":0.625},"last":0.318}]},{"osd":1,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.47399999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.57399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.376},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73499999999999999,"5min":1.006,"15min":1.006},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:55:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.748,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71999999999999997,"15min":0.83599999999999997},"min":{"1min":0.41199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78600000000000003,"5min":4.9489999999999998,"15min":4.9489999999999998},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":0.876,"15min":0.876},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.59399999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68999999999999995,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.51300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88600000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.63700000000000001,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67500000000000004,"5min":1.226,"15min":1.226},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67700000000000005,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81399999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94999999999999996,"5min":1.266,"15min":1.266},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.99299999999999999,"5min":1.087,"15min":1.087},"last":0.57499999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":498753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52700000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.54300000000000004,"5min":0.251,"15min":0.251},"max":{"1min":0.80900000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57299999999999995,"15min":0.52400000000000002},"min":{"1min":0.51400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.376},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.43099999999999999},"min":{"1min":0.33600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42099999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.54600000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84399999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.66600000000000004,"15min":0.60799999999999998},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78800000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68200000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73299999999999998,"15min":0.71799999999999997},"min":{"1min":0.51800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.72099999999999997},"min":{"1min":0.57599999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.749},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.80500000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73499999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":496017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.47599999999999998},"min":{"1min":0.34999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82399999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.48299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.502},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82599999999999996,"5min":2.206,"15min":2.206},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.752,"5min":1.04,"15min":1.04},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79600000000000004,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.255,"15min":0.255},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.34799999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61799999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.442}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.501},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65400000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.70099999999999996,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84399999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:55:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.39500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74299999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60599999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89900000000000002,"5min":0.996,"15min":0.996},"last":0.73799999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54500000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":495777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.49399999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71499999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56399999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.57599999999999996,"15min":0.52400000000000002},"min":{"1min":0.51000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.61099999999999999,"15min":0.53600000000000003},"min":{"1min":0.49299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.92400000000000004,"5min":2.698,"15min":2.698},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.49099999999999999},"min":{"1min":0.371,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65800000000000003,"15min":0.54600000000000004},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.51,"5min":8.282,"15min":8.282},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.98199999999999998,"5min":1.024,"15min":1.024},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.499,"15min":0.45700000000000002},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.88500000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.45500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6899999999999999,"5min":8.7949999999999999,"15min":8.7949999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:55:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.94299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.92700000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84099999999999997,"15min":0.73799999999999999},"min":{"1min":0.71999999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.718,"5min":8.2279999999999998,"15min":8.2279999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85999999999999999,"15min":0.76000000000000001},"min":{"1min":0.66200000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7310000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.70299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.73499999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.008}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55500000000000005,"5min":0.621,"15min":0.621},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60499999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.309}]},{"osd":1,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56799999999999995,"15min":0.53400000000000003},"min":{"1min":0.438,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81499999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.442}]},{"osd":2,"last update":"Sun Oct 8 09:55:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.57499999999999996},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.53600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45400000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:55:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:55:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":1.214,"15min":1.214},"last":0.625},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55200000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65900000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:55:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5170000000000003,"5min":6.5170000000000003,"15min":6.5170000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.60499999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86699999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58199999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64400000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:52:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.752}]},{"osd":1,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.316,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33400000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56100000000000005,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.47699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.53600000000000003},"min":{"1min":0.44900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86099999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55700000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.47599999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.877,"15min":0.877},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.80500000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":1.01,"15min":1.01},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86899999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.80700000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.495}]},{"osd":8,"last update":"Sun Oct 8 09:55:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.56599999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.014,"15min":1.014},"last":0.747},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999}]},{"osd":9,"last update":"Sun Oct 8 09:55:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92500000000000004,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.90300000000000002,"15min":0.95299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0049999999999999,"5min":5.492,"15min":5.492},"last":0.86299999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91500000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.006,"5min":1.048,"15min":1.048},"last":0.89800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:54:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79600000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67100000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89300000000000002,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":1.379,"15min":1.379},"last":0.84399999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77600000000000002,"5min":1.407,"15min":1.407},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46700000000000003,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67200000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.73399999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":1.498,"15min":1.498},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.871,"5min":1.476,"15min":1.476},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.48599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92700000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":1.165,"15min":1.165},"last":0.90100000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.67500000000000004,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0269999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78400000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94099999999999995,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.008},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.621},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.69999999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:55:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.97399999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:54:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94099999999999995,"5min":1.367,"15min":1.367},"last":0.874},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1020000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:55:54.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 -- 172.21.15.97:0/897110148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85881001b0 msgr2=0x7f8588107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 --2- 172.21.15.97:0/897110148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85881001b0 0x7f8588107a50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f858000b0a0 tx=0x7f858000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 -- 172.21.15.97:0/897110148 shutdown_connections 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 --2- 172.21.15.97:0/897110148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85881001b0 0x7f8588107a50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 --2- 172.21.15.97:0/897110148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8588100c30 0x7f8588101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 --2- 172.21.15.97:0/897110148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 -- 172.21.15.97:0/897110148 >> 172.21.15.97:0/897110148 conn(0x7f85880fa2c0 msgr2=0x7f85880fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 -- 172.21.15.97:0/897110148 shutdown_connections 2023-10-08T09:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 -- 172.21.15.97:0/897110148 wait complete. 2023-10-08T09:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.029+0000 7f858cc79700 1 Processor -- start 2023-10-08T09:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 -- start start 2023-10-08T09:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 0x7f858819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819e240 con 0x7f8588100c30 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819e3b0 con 0x7f85880ff7f0 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819e570 con 0x7f85881001b0 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57070/0 (socket says 172.21.15.97:57070) 2023-10-08T09:55:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 -- 172.21.15.97:0/4059478566 learned_addr learned my addr 172.21.15.97:0/4059478566 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8585d9b700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 0x7f858819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858659c700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 -- 172.21.15.97:0/4059478566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 msgr2=0x7f858819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 0x7f858819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 -- 172.21.15.97:0/4059478566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 msgr2=0x7f85880ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 -- 172.21.15.97:0/4059478566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009d00 con 0x7f8588100c30 2023-10-08T09:55:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8585d9b700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 0x7f858819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858659c700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f8586d9d700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8580004d00 tx=0x7f8580004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858000e040 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.030+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858819e790 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.031+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858001db50 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.031+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858001bbe0 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.031+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858819ec10 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.031+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85881069c0 con 0x7f8588100c30 2023-10-08T09:55:55.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.033+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f858001bd40 con 0x7f8588100c30 2023-10-08T09:55:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.033+0000 7f85777fe700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 0x7f85700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.033+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f85800cf3d0 con 0x7f8588100c30 2023-10-08T09:55:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.033+0000 7f858659c700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 0x7f85700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.034+0000 7f858659c700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 0x7f85700a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f857c005fd0 tx=0x7f857c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.037+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8580098790 con 0x7f8588100c30 2023-10-08T09:55:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.231+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8588066bc0 con 0x7f8588100c30 2023-10-08T09:55:55.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.233+0000 7f85777fe700 1 -- 172.21.15.97:0/4059478566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f858009bdb0 con 0x7f8588100c30 2023-10-08T09:55:55.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:55.235 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:55.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 msgr2=0x7f85700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 0x7f85700a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f857c005fd0 tx=0x7f857c005ee0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 msgr2=0x7f858819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8580004d00 tx=0x7f8580004de0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 shutdown_connections 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f85700a1480 0x7f85700a3940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8588100c30 0x7f858819da50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85880ff7f0 0x7f85880ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 --2- 172.21.15.97:0/4059478566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85881001b0 0x7f858819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 >> 172.21.15.97:0/4059478566 conn(0x7f85880fa2c0 msgr2=0x7f8588103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 shutdown_connections 2023-10-08T09:55:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.235+0000 7f858cc79700 1 -- 172.21.15.97:0/4059478566 wait complete. 2023-10-08T09:55:55.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:55:55.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:55:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.372+0000 7fd9b7578700 1 -- 172.21.15.97:0/1946730617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b0102490 msgr2=0x7fd9b0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.372+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1946730617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b0102490 0x7fd9b0108d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b0a0 tx=0x7fd9a800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- 172.21.15.97:0/1946730617 shutdown_connections 2023-10-08T09:55:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1946730617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b0102490 0x7fd9b0108d40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1946730617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b00fea80 0x7fd9b00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1946730617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b0101af0 0x7fd9b0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- 172.21.15.97:0/1946730617 >> 172.21.15.97:0/1946730617 conn(0x7fd9b00fa390 msgr2=0x7fd9b00fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- 172.21.15.97:0/1946730617 shutdown_connections 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- 172.21.15.97:0/1946730617 wait complete. 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 Processor -- start 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- start start 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b0101af0 0x7fd9b00ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b0102490 0x7fd9b00fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0100780 con 0x7fd9b00fea80 2023-10-08T09:55:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b01008f0 con 0x7fd9b0102490 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.373+0000 7fd9b7578700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b01a0860 con 0x7fd9b0101af0 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57080/0 (socket says 172.21.15.97:57080) 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 -- 172.21.15.97:0/1709361709 learned_addr learned my addr 172.21.15.97:0/1709361709 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b4b13700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b0101af0 0x7fd9b00ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 -- 172.21.15.97:0/1709361709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b0101af0 msgr2=0x7fd9b00ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b0101af0 0x7fd9b00ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 -- 172.21.15.97:0/1709361709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b0102490 msgr2=0x7fd9b00fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b0102490 0x7fd9b00fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 -- 172.21.15.97:0/1709361709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8009d00 con 0x7fd9b00fea80 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b5314700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00c970 tx=0x7fd9ac00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac004220 con 0x7fd9b00fea80 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b01a0af0 con 0x7fd9b00fea80 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9ac007d60 con 0x7fd9b00fea80 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac003d00 con 0x7fd9b00fea80 2023-10-08T09:55:55.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.374+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b01a1000 con 0x7fd9b00fea80 2023-10-08T09:55:55.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.375+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b0107d30 con 0x7fd9b00fea80 2023-10-08T09:55:55.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.376+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd9ac004380 con 0x7fd9b00fea80 2023-10-08T09:55:55.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.377+0000 7fd9a67fc700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.377+0000 7fd9b4b13700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.377+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd9ac0cd020 con 0x7fd9b00fea80 2023-10-08T09:55:55.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.377+0000 7fd9b4b13700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 0x7fd99c0a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005950 tx=0x7fd9a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.381+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9ac096350 con 0x7fd9b00fea80 2023-10-08T09:55:55.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:55 smithi186 ceph-mon[104393]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:55:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.570+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9b010bfa0 con 0x7fd9b00fea80 2023-10-08T09:55:55.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.571+0000 7fd9a67fc700 1 -- 172.21.15.97:0/1709361709 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9ac01d0a0 con 0x7fd9b00fea80 2023-10-08T09:55:55.573 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T09:55:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.572+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 msgr2=0x7fd99c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.572+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 0x7fd99c0a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005950 tx=0x7fd9a00058e0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.572+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 msgr2=0x7fd9b0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.572+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00c970 tx=0x7fd9ac00cd30 comp rx=0 tx=0).stop 2023-10-08T09:55:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 shutdown_connections 2023-10-08T09:55:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b0102490 0x7fd9b00fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b00fea80 0x7fd9b0101410 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 --2- 172.21.15.97:0/1709361709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9b0101af0 0x7fd9b00ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 >> 172.21.15.97:0/1709361709 conn(0x7fd9b00fa390 msgr2=0x7fd9b0104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 shutdown_connections 2023-10-08T09:55:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.573+0000 7fd9b7578700 1 -- 172.21.15.97:0/1709361709 wait complete. 2023-10-08T09:55:55.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:55 smithi097 ceph-mon[99491]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:55:55.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:55:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:55 smithi178 ceph-mon[103156]: pgmap v323: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 -- 172.21.15.97:0/2001078027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1019b0 msgr2=0x7f8f6c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 --2- 172.21.15.97:0/2001078027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c101e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0099c0 tx=0x7f8f5c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 -- 172.21.15.97:0/2001078027 shutdown_connections 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 --2- 172.21.15.97:0/2001078027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c102350 0x7f8f6c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 --2- 172.21.15.97:0/2001078027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 --2- 172.21.15.97:0/2001078027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c101e10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.709+0000 7f8f732b2700 1 -- 172.21.15.97:0/2001078027 >> 172.21.15.97:0/2001078027 conn(0x7f8f6c0fbd30 msgr2=0x7f8f6c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- 172.21.15.97:0/2001078027 shutdown_connections 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- 172.21.15.97:0/2001078027 wait complete. 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 Processor -- start 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- start start 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c102350 0x7f8f6c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c19e230 con 0x7f8f6c1007c0 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c19e370 con 0x7f8f6c102350 2023-10-08T09:55:55.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f732b2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c19e500 con 0x7f8f6c1019b0 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f7104e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f7104e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57094/0 (socket says 172.21.15.97:57094) 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.710+0000 7f8f7104e700 1 -- 172.21.15.97:0/36674417 learned_addr learned my addr 172.21.15.97:0/36674417 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7184f700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c102350 0x7f8f6c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7084d700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 -- 172.21.15.97:0/36674417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c1019b0 msgr2=0x7f8f6c196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 -- 172.21.15.97:0/36674417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c102350 msgr2=0x7f8f6c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c102350 0x7f8f6c19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 -- 172.21.15.97:0/36674417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0095f0 con 0x7f8f6c1007c0 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f7104e700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800d980 tx=0x7f8f6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68004020 con 0x7f8f6c1007c0 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f68009730 con 0x7f8f6c1007c0 2023-10-08T09:55:55.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c19e7e0 con 0x7f8f6c1007c0 2023-10-08T09:55:55.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68003a80 con 0x7f8f6c1007c0 2023-10-08T09:55:55.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.711+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c19ed40 con 0x7f8f6c1007c0 2023-10-08T09:55:55.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.712+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c107c10 con 0x7f8f6c1007c0 2023-10-08T09:55:55.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.713+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8f680098a0 con 0x7f8f6c1007c0 2023-10-08T09:55:55.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.713+0000 7f8f627fc700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 0x7f8f580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:55.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.714+0000 7f8f7084d700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 0x7f8f580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:55.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.714+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8f680cd030 con 0x7f8f6c1007c0 2023-10-08T09:55:55.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.714+0000 7f8f7084d700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 0x7f8f580a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:55.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.717+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f68096380 con 0x7f8f6c1007c0 2023-10-08T09:55:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.911+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f6c10c290 con 0x7f8f6c1007c0 2023-10-08T09:55:55.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.911+0000 7f8f627fc700 1 -- 172.21.15.97:0/36674417 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f680999a0 con 0x7f8f6c1007c0 2023-10-08T09:55:55.913 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:55.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 msgr2=0x7f8f580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 0x7f8f580a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000c00 tx=0x7f8f5c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 msgr2=0x7f8f6c196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:55.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800d980 tx=0x7f8f6800dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 shutdown_connections 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f8f580a1500 0x7f8f580a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6c102350 0x7f8f6c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f6c1007c0 0x7f8f6c196700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 --2- 172.21.15.97:0/36674417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6c1019b0 0x7f8f6c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 >> 172.21.15.97:0/36674417 conn(0x7f8f6c0fbd30 msgr2=0x7f8f6c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 shutdown_connections 2023-10-08T09:55:55.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:55.913+0000 7f8f732b2700 1 -- 172.21.15.97:0/36674417 wait complete. 2023-10-08T09:55:55.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:55:56.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 -- 172.21.15.97:0/3145444465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fee50 msgr2=0x7f86980ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 --2- 172.21.15.97:0/3145444465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fee50 0x7f86980ff270 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f868c009990 tx=0x7f868c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 -- 172.21.15.97:0/3145444465 shutdown_connections 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 --2- 172.21.15.97:0/3145444465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8698100960 0x7f8698107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 --2- 172.21.15.97:0/3145444465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fee50 0x7f86980ff270 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 --2- 172.21.15.97:0/3145444465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86980fffa0 0x7f8698100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 -- 172.21.15.97:0/3145444465 >> 172.21.15.97:0/3145444465 conn(0x7f86980fa3c0 msgr2=0x7f86980fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 -- 172.21.15.97:0/3145444465 shutdown_connections 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.049+0000 7f869efd0700 1 -- 172.21.15.97:0/3145444465 wait complete. 2023-10-08T09:55:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 Processor -- start 2023-10-08T09:55:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 -- start start 2023-10-08T09:55:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86980fee50 0x7f8698196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8698100960 0x7f869819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e270 con 0x7f86980fffa0 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e3b0 con 0x7f86980fee50 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.050+0000 7f869efd0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e540 con 0x7f8698100960 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57110/0 (socket says 172.21.15.97:57110) 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 -- 172.21.15.97:0/2482690704 learned_addr learned my addr 172.21.15.97:0/2482690704 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f869cd6c700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86980fee50 0x7f8698196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 -- 172.21.15.97:0/2482690704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8698100960 msgr2=0x7f869819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8698100960 0x7f869819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 -- 172.21.15.97:0/2482690704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86980fee50 msgr2=0x7f8698196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86980fee50 0x7f8698196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 -- 172.21.15.97:0/2482690704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0095f0 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8697fff700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f868400d980 tx=0x7f868400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684004a30 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869819e820 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86840054e0 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684009c90 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.051+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869819ece0 con 0x7f86980fffa0 2023-10-08T09:55:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.052+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86981062b0 con 0x7f86980fffa0 2023-10-08T09:55:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.053+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8684004c20 con 0x7f86980fffa0 2023-10-08T09:55:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.054+0000 7f8695ffb700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 0x7f86880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.054+0000 7f869cd6c700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 0x7f86880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.054+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86840ccf10 con 0x7f86980fffa0 2023-10-08T09:55:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.054+0000 7f869cd6c700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 0x7f86880a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b5c0 tx=0x7f868c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.058+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8684096240 con 0x7f86980fffa0 2023-10-08T09:55:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.256+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8698105c90 con 0x7f86980fffa0 2023-10-08T09:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.256+0000 7f8695ffb700 1 -- 172.21.15.97:0/2482690704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86840170a0 con 0x7f86980fffa0 2023-10-08T09:55:56.258 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T09:55:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 msgr2=0x7f86880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 0x7f86880a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b5c0 tx=0x7f868c003790 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 msgr2=0x7f8698196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f868400d980 tx=0x7f868400dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 shutdown_connections 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f86880a1480 0x7f86880a3940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8698100960 0x7f869819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86980fee50 0x7f8698196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 --2- 172.21.15.97:0/2482690704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980fffa0 0x7f8698196c80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 >> 172.21.15.97:0/2482690704 conn(0x7f86980fa3c0 msgr2=0x7f86981033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 shutdown_connections 2023-10-08T09:55:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.258+0000 7f869efd0700 1 -- 172.21.15.97:0/2482690704 wait complete. 2023-10-08T09:55:56.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 -- 172.21.15.97:0/2127381694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 msgr2=0x7fde700ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 --2- 172.21.15.97:0/2127381694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde700ffd60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fde60009990 tx=0x7fde60009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 -- 172.21.15.97:0/2127381694 shutdown_connections 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 --2- 172.21.15.97:0/2127381694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde701002a0 0x7fde70107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 --2- 172.21.15.97:0/2127381694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde70100d20 0x7fde70101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 --2- 172.21.15.97:0/2127381694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde700ffd60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 -- 172.21.15.97:0/2127381694 >> 172.21.15.97:0/2127381694 conn(0x7fde700fa390 msgr2=0x7fde700fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 -- 172.21.15.97:0/2127381694 shutdown_connections 2023-10-08T09:55:56.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.396+0000 7fde76850700 1 -- 172.21.15.97:0/2127381694 wait complete. 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 Processor -- start 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 -- start start 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 0x7fde70198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 0x7fde7019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701a0440 con 0x7fde700ff8e0 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701a0580 con 0x7fde70100d20 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde76850700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701a0710 con 0x7fde701002a0 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57122/0 (socket says 172.21.15.97:57122) 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 -- 172.21.15.97:0/1654920408 learned_addr learned my addr 172.21.15.97:0/1654920408 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:56.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde74ded700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 0x7fde7019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6f7fe700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 0x7fde70198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 -- 172.21.15.97:0/1654920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 msgr2=0x7fde70198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 0x7fde70198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 -- 172.21.15.97:0/1654920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 msgr2=0x7fde7019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 0x7fde7019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 -- 172.21.15.97:0/1654920408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600095f0 con 0x7fde700ff8e0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6f7fe700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 0x7fde70198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde74ded700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 0x7fde7019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.397+0000 7fde6ffff700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fde6400d980 tx=0x7fde6400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.398+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64009b40 con 0x7fde700ff8e0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.398+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde6400de60 con 0x7fde700ff8e0 2023-10-08T09:55:56.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.398+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde701a09f0 con 0x7fde700ff8e0 2023-10-08T09:55:56.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.398+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64005580 con 0x7fde700ff8e0 2023-10-08T09:55:56.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.398+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde701a0eb0 con 0x7fde700ff8e0 2023-10-08T09:55:56.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.399+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde70066ca0 con 0x7fde700ff8e0 2023-10-08T09:55:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.400+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fde64004750 con 0x7fde700ff8e0 2023-10-08T09:55:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.401+0000 7fde6d7fa700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.401+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fde640cd420 con 0x7fde700ff8e0 2023-10-08T09:55:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.401+0000 7fde6f7fe700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.401+0000 7fde6f7fe700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 0x7fde580a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fde600037f0 tx=0x7fde60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.405+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde64096750 con 0x7fde700ff8e0 2023-10-08T09:55:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:56 smithi186 ceph-mon[104393]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:56 smithi186 ceph-mon[104393]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4059478566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:56.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1709361709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/36674417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.596+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fde7010b050 con 0x7fde700ff8e0 2023-10-08T09:55:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.596+0000 7fde6d7fa700 1 -- 172.21.15.97:0/1654920408 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fde64017070 con 0x7fde700ff8e0 2023-10-08T09:55:56.598 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T09:55:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 msgr2=0x7fde580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 0x7fde580a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fde600037f0 tx=0x7fde60005fb0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 msgr2=0x7fde70198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fde6400d980 tx=0x7fde6400dd40 comp rx=0 tx=0).stop 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 shutdown_connections 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde70100d20 0x7fde7019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde700ff8e0 0x7fde70198910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 --2- 172.21.15.97:0/1654920408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fde701002a0 0x7fde70198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 >> 172.21.15.97:0/1654920408 conn(0x7fde700fa390 msgr2=0x7fde700fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 shutdown_connections 2023-10-08T09:55:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.598+0000 7fde76850700 1 -- 172.21.15.97:0/1654920408 wait complete. 2023-10-08T09:55:56.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:56 smithi097 ceph-mon[99491]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:56 smithi097 ceph-mon[99491]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4059478566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:56.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1709361709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.613 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/36674417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:55:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:56 smithi178 ceph-mon[103156]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:56 smithi178 ceph-mon[103156]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:55:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4059478566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:56.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1709361709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/36674417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:55:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 -- 172.21.15.97:0/3014549484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fed50 msgr2=0x7ff1500ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 --2- 172.21.15.97:0/3014549484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fed50 0x7ff1500ff170 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff140009990 tx=0x7ff140009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 -- 172.21.15.97:0/3014549484 shutdown_connections 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 --2- 172.21.15.97:0/3014549484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1501008e0 0x7ff150107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 --2- 172.21.15.97:0/3014549484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fed50 0x7ff1500ff170 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 --2- 172.21.15.97:0/3014549484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1500fff40 0x7ff1501003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 -- 172.21.15.97:0/3014549484 >> 172.21.15.97:0/3014549484 conn(0x7ff1500fa2c0 msgr2=0x7ff1500fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.732+0000 7ff158300700 1 -- 172.21.15.97:0/3014549484 shutdown_connections 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 -- 172.21.15.97:0/3014549484 wait complete. 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 Processor -- start 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 -- start start 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1500fed50 0x7ff150196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1501008e0 0x7ff15019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019e130 con 0x7ff1500fff40 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019e270 con 0x7ff1500fed50 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff158300700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019e400 con 0x7ff1501008e0 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff15589b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff15609c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1500fed50 0x7ff150196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff15589b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57138/0 (socket says 172.21.15.97:57138) 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff15589b700 1 -- 172.21.15.97:0/4002641518 learned_addr learned my addr 172.21.15.97:0/4002641518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.733+0000 7ff15689d700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1501008e0 0x7ff15019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 -- 172.21.15.97:0/4002641518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1501008e0 msgr2=0x7ff15019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1501008e0 0x7ff15019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 -- 172.21.15.97:0/4002641518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1500fed50 msgr2=0x7ff150196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1500fed50 0x7ff150196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 -- 172.21.15.97:0/4002641518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400095f0 con 0x7ff1500fff40 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff15589b700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d980 tx=0x7ff14c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c004a30 con 0x7ff1500fff40 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15019e6e0 con 0x7ff1500fff40 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c0054e0 con 0x7ff1500fff40 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00fbc0 con 0x7ff1500fff40 2023-10-08T09:55:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.734+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15019eba0 con 0x7ff1500fff40 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.735+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150106180 con 0x7ff1500fff40 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.736+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff14c004c20 con 0x7ff1500fff40 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.737+0000 7ff1477fe700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 0x7ff13c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.737+0000 7ff15609c700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 0x7ff13c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.737+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff14c0cd270 con 0x7ff1500fff40 2023-10-08T09:55:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.737+0000 7ff15609c700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 0x7ff13c0a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b5c0 tx=0x7ff140003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:56.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.740+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff14c0965a0 con 0x7ff1500fff40 2023-10-08T09:55:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.984+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1500678a0 con 0x7ff1500fff40 2023-10-08T09:55:56.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.984+0000 7ff1477fe700 1 -- 172.21.15.97:0/4002641518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7ff14c017070 con 0x7ff1500fff40 2023-10-08T09:55:56.986 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:56.986 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 msgr2=0x7ff13c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 0x7ff13c0a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b5c0 tx=0x7ff140003790 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 msgr2=0x7ff150196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00d980 tx=0x7ff14c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 shutdown_connections 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff13c0a1480 0x7ff13c0a3940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1501008e0 0x7ff15019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1500fed50 0x7ff150196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 --2- 172.21.15.97:0/4002641518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1500fff40 0x7ff150196b40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:56.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 >> 172.21.15.97:0/4002641518 conn(0x7ff1500fa2c0 msgr2=0x7ff1501032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:56.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 shutdown_connections 2023-10-08T09:55:56.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:56.987+0000 7ff158300700 1 -- 172.21.15.97:0/4002641518 wait complete. 2023-10-08T09:55:56.990 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:55:57.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:55:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 -- 172.21.15.97:0/4040016151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101120 msgr2=0x7fdbf4103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/4040016151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101120 0x7fdbf4103510 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009990 tx=0x7fdbe4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 -- 172.21.15.97:0/4040016151 shutdown_connections 2023-10-08T09:55:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/4040016151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4106380 0x7fdbf41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/4040016151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4101120 0x7fdbf4103510 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/4040016151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4103a50 0x7fdbf4105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 -- 172.21.15.97:0/4040016151 >> 172.21.15.97:0/4040016151 conn(0x7fdbf40fab10 msgr2=0x7fdbf40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 -- 172.21.15.97:0/4040016151 shutdown_connections 2023-10-08T09:55:57.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.121+0000 7fdbfa1db700 1 -- 172.21.15.97:0/4040016151 wait complete. 2023-10-08T09:55:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 Processor -- start 2023-10-08T09:55:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 -- start start 2023-10-08T09:55:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 0x7fdbf41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4103a50 0x7fdbf4196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf419e0d0 con 0x7fdbf4106380 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf419e210 con 0x7fdbf4101120 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbfa1db700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf419e3a0 con 0x7fdbf4103a50 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57154/0 (socket says 172.21.15.97:57154) 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.122+0000 7fdbf3fff700 1 -- 172.21.15.97:0/2968314497 learned_addr learned my addr 172.21.15.97:0/2968314497 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf2ffd700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4103a50 0x7fdbf4196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf37fe700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 0x7fdbf41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 -- 172.21.15.97:0/2968314497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4103a50 msgr2=0x7fdbf4196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4103a50 0x7fdbf4196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 -- 172.21.15.97:0/2968314497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 msgr2=0x7fdbf41965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 0x7fdbf41965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 -- 172.21.15.97:0/2968314497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe40095f0 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf37fe700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 0x7fdbf41965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf3fff700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdbec007ee0 tx=0x7fdbec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec017070 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf419e680 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbec00f040 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec005740 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.123+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf40ff180 con 0x7fdbf4106380 2023-10-08T09:55:57.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.124+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf40fc690 con 0x7fdbf4106380 2023-10-08T09:55:57.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.125+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdbec00dc60 con 0x7fdbf4106380 2023-10-08T09:55:57.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.126+0000 7fdbf0ff9700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 0x7fdbdc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.126+0000 7fdbf37fe700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 0x7fdbdc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.126+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbec0cc7d0 con 0x7fdbf4106380 2023-10-08T09:55:57.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.126+0000 7fdbf37fe700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 0x7fdbdc0a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b2e0 tx=0x7fdbe400b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.130+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdbec095b00 con 0x7fdbf4106380 2023-10-08T09:55:57.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.327+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbf4067450 con 0x7fdbf4106380 2023-10-08T09:55:57.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.328+0000 7fdbf0ff9700 1 -- 172.21.15.97:0/2968314497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7fdbec099120 con 0x7fdbf4106380 2023-10-08T09:55:57.331 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:55:57.331 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 msgr2=0x7fdbdc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 0x7fdbdc0a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b2e0 tx=0x7fdbe400b770 comp rx=0 tx=0).stop 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 msgr2=0x7fdbf419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdbec007ee0 tx=0x7fdbec00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 shutdown_connections 2023-10-08T09:55:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdbdc0a1480 0x7fdbdc0a3940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbf4106380 0x7fdbf419db00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf4101120 0x7fdbf41965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 --2- 172.21.15.97:0/2968314497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbf4103a50 0x7fdbf4196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 >> 172.21.15.97:0/2968314497 conn(0x7fdbf40fab10 msgr2=0x7fdbf4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 shutdown_connections 2023-10-08T09:55:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.331+0000 7fdbfa1db700 1 -- 172.21.15.97:0/2968314497 wait complete. 2023-10-08T09:55:57.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:57 smithi097 ceph-mon[99491]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2023-10-08T09:55:57.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2482690704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1654920408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.344 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/4002641518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:57.346 DEBUG:tasks.fwd_scrub:Yielding 2023-10-08T09:55:57.346 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-08T09:55:57.357 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T09:55:57.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-08T09:55:57.358 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-08T09:55:57.358 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:55:57.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:55:57.360 INFO:teuthology.run_tasks:Running task workunit... 2023-10-08T09:55:57.448 INFO:tasks.workunit:Pulling workunits from ref 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:55:57.448 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-08T09:55:57.448 INFO:tasks.workunit:timeout=3h 2023-10-08T09:55:57.448 INFO:tasks.workunit:cleanup=True 2023-10-08T09:55:57.449 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 7ch/124d Inode: 1 Links: 2 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-10-08 09:53:57.200991720 +0000 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-10-08 09:54:08.758942794 +0000 2023-10-08T09:55:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-10-08 09:53:57.200991720 +0000 2023-10-08T09:55:57.467 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-08T09:55:57.468 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-08T09:55:57.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 -- 172.21.15.97:0/3497574835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102350 msgr2=0x7fe710108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 --2- 172.21.15.97:0/3497574835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102350 0x7fe710108c00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe70800b0d0 tx=0x7fe70800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 -- 172.21.15.97:0/3497574835 shutdown_connections 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 --2- 172.21.15.97:0/3497574835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102350 0x7fe710108c00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 --2- 172.21.15.97:0/3497574835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7101007c0 0x7fe710100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 --2- 172.21.15.97:0/3497574835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101019b0 0x7fe710101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 -- 172.21.15.97:0/3497574835 >> 172.21.15.97:0/3497574835 conn(0x7fe7100fbd30 msgr2=0x7fe7100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 -- 172.21.15.97:0/3497574835 shutdown_connections 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.485+0000 7fe7187c7700 1 -- 172.21.15.97:0/3497574835 wait complete. 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 Processor -- start 2023-10-08T09:55:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 -- start start 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 0x7fe710198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 0x7fe71019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101a0550 con 0x7fe7101019b0 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101a0690 con 0x7fe7101007c0 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe7187c7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101a0820 con 0x7fe710102350 2023-10-08T09:55:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57166/0 (socket says 172.21.15.97:57166) 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 -- 172.21.15.97:0/1744138807 learned_addr learned my addr 172.21.15.97:0/1744138807 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe716563700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 0x7fe710198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe716d64700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 0x7fe71019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 -- 172.21.15.97:0/1744138807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 msgr2=0x7fe71019ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 0x7fe71019ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 -- 172.21.15.97:0/1744138807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 msgr2=0x7fe710198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 0x7fe710198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe715d62700 1 -- 172.21.15.97:0/1744138807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708009d00 con 0x7fe7101019b0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe716d64700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 0x7fe71019ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.486+0000 7fe716563700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 0x7fe710198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe715d62700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe7077fe700 1 -- 172.21.15.97:0/1744138807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c009b40 con 0x7fe7101019b0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7101a0b00 con 0x7fe7101019b0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe7077fe700 1 -- 172.21.15.97:0/1744138807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70c009ca0 con 0x7fe7101019b0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe7077fe700 1 -- 172.21.15.97:0/1744138807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00f620 con 0x7fe7101019b0 2023-10-08T09:55:57.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.487+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7101a1060 con 0x7fe7101019b0 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.489+0000 7fe7077fe700 1 -- 172.21.15.97:0/1744138807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe70c010460 con 0x7fe7101019b0 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.489+0000 7fe7077fe700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 0x7fe6fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.490+0000 7fe716563700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 0x7fe6fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.490+0000 7fe7077fe700 1 -- 172.21.15.97:0/1744138807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe70c0cd2e0 con 0x7fe7101019b0 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.490+0000 7fe716563700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 0x7fe6fc0a3940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe710101810 tx=0x7fe70000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.490+0000 7fe7249b0b80 1 Processor -- start 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.490+0000 7fe7249b0b80 1 -- start start 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d911042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d91104d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9106b4e0 con 0x559d911030e0 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9105cca0 con 0x559d91103a40 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9105a5c0 con 0x559d91104880 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57172/0 (socket says 172.21.15.97:57172) 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 -- 172.21.15.97:0/168522579 learned_addr learned my addr 172.21.15.97:0/168522579 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 -- 172.21.15.97:0/168522579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 msgr2=0x559d91104d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d91104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe715d62700 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d911042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 -- 172.21.15.97:0/168522579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 msgr2=0x559d911042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d911042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 -- 172.21.15.97:0/168522579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d90e80e60 con 0x559d911030e0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe716563700 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe70000ed90 tx=0x7fe700011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f8503652d4d89c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe704ff9700 1 -- 172.21.15.97:0/168522579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700010e50 con 0x559d911030e0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe704ff9700 1 -- 172.21.15.97:0/168522579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70001d910 con 0x559d911030e0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.491+0000 7fe704ff9700 1 -- 172.21.15.97:0/168522579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70000f990 con 0x559d911030e0 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 -- 172.21.15.97:0/168522579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 msgr2=0x559d91103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe70000ed90 tx=0x7fe700011000 comp rx=0 tx=0).stop 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 -- 172.21.15.97:0/168522579 shutdown_connections 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d91104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d91103500 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/168522579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d911042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 -- 172.21.15.97:0/168522579 >> 172.21.15.97:0/168522579 conn(0x559d910772e0 msgr2=0x559d910fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 -- 172.21.15.97:0/168522579 shutdown_connections 2023-10-08T09:55:57.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.492+0000 7fe7249b0b80 1 -- 172.21.15.97:0/168522579 wait complete. 2023-10-08T09:55:57.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 -- start start 2023-10-08T09:55:57.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 Processor -- start 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d91055eb0 con 0x559d91104880 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d90ff2790 con 0x559d91103a40 2023-10-08T09:55:57.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d90ff2430 con 0x559d911030e0 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe7249b0b80 10 client.0 _ll_register_callbacks cb 0x559d90e4f880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57182/0 (socket says 172.21.15.97:57182) 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 -- 172.21.15.97:0/3966032112 learned_addr learned my addr 172.21.15.97:0/3966032112 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe715d62700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716563700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 -- 172.21.15.97:0/3966032112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 msgr2=0x559d9110ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 -- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 msgr2=0x559d9110d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 -- 172.21.15.97:0/3966032112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70000ae70 con 0x559d91104880 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe715d62700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe716d64700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe70800b820 tx=0x7fe708007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ebf53662b38e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.493+0000 7fe6f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:57.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800e050 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7080092d0 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d9101df30 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe708009490 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 10 client.15705 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d90e69ce0 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d9101df30 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe70800b100 con 0x559d91104880 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:55:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 20 client.15705 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d91118c60 0x559d9111b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 4 client.15705 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d9104ee90 con 0x559d91118c60 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.494+0000 7fe716563700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d91118c60 0x559d9111b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.495+0000 7fe716563700 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d91118c60 0x559d9111b080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a8e32c851840e server_cookie=cde91115c9209610 in_seq=0 out_seq=0 2023-10-08T09:55:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.495+0000 7fe6f9ffb700 10 client.15705 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3966032112 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d9104ee90 con 0x559d91118c60 2023-10-08T09:55:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe6f9ffb700 10 client.15705 handle_command_reply: tid=0 2023-10-08T09:55:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 client.15705 shutdown 2023-10-08T09:55:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 msgr2=0x559d911143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe70800b820 tx=0x7fe708007590 comp rx=0 tx=0).stop 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 shutdown_connections 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d911143e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d91118c60 0x559d9111b080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe6f9ffb700 0 client.15705 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3966032112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 >> 172.21.15.97:0/3966032112 conn(0x559d910772e0 msgr2=0x559d910fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 shutdown_connections 2023-10-08T09:55:57.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3966032112 wait complete. 2023-10-08T09:55:57.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.496+0000 7fe7249b0b80 20 client.15705 trim_cache size 0 max 16384 2023-10-08T09:55:57.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.513+0000 7fe7249b0b80 1 Processor -- start 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.513+0000 7fe7249b0b80 1 -- start start 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d90ed69a0 con 0x559d91104880 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9110c9d0 con 0x559d91103a40 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d91114870 con 0x559d911030e0 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716d64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:58886/0 (socket says 172.21.15.97:58886) 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716d64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57190/0 (socket says 172.21.15.97:57190) 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 -- 172.21.15.97:0/1195702023 learned_addr learned my addr 172.21.15.97:0/1195702023 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe715d62700 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 -- 172.21.15.97:0/1195702023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 msgr2=0x559d9110d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 -- 172.21.15.97:0/1195702023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 msgr2=0x559d91114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 -- 172.21.15.97:0/1195702023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d9111fcb0 con 0x559d911030e0 2023-10-08T09:55:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716d64700 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe716563700 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe70000a5c0 tx=0x7fe70000aa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7be109d1f74922b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.514+0000 7fe7057fa700 1 -- 172.21.15.97:0/1195702023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7000323b0 con 0x559d911030e0 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7057fa700 1 -- 172.21.15.97:0/1195702023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70002d040 con 0x559d911030e0 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7057fa700 1 -- 172.21.15.97:0/1195702023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700004280 con 0x559d911030e0 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 -- 172.21.15.97:0/1195702023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 msgr2=0x559d9110ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe70000a5c0 tx=0x7fe70000aa20 comp rx=0 tx=0).stop 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 -- 172.21.15.97:0/1195702023 shutdown_connections 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d91104880 0x559d91114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d911030e0 0x559d9110ce60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/1195702023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d9110d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 -- 172.21.15.97:0/1195702023 >> 172.21.15.97:0/1195702023 conn(0x559d910772e0 msgr2=0x559d91107820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 -- 172.21.15.97:0/1195702023 shutdown_connections 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.515+0000 7fe7249b0b80 1 -- 172.21.15.97:0/1195702023 wait complete. 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 -- start start 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 Processor -- start 2023-10-08T09:55:57.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d91113c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d9116ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9116d380 con 0x559d911030e0 2023-10-08T09:55:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9116d4c0 con 0x559d91103a40 2023-10-08T09:55:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d9116d680 con 0x559d91104880 2023-10-08T09:55:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe7249b0b80 10 client.0 _ll_register_callbacks cb 0x7fe708009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57192/0 (socket says 172.21.15.97:57192) 2023-10-08T09:55:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 -- 172.21.15.97:0/3302228849 learned_addr learned my addr 172.21.15.97:0/3302228849 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716d64700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d9116ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe715d62700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d91113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 -- 172.21.15.97:0/3302228849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 msgr2=0x559d9116ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d9116ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.516+0000 7fe716563700 1 -- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 msgr2=0x559d91113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe716563700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d91113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe716563700 1 -- 172.21.15.97:0/3302228849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d90e69ce0 con 0x559d911030e0 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe716d64700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d9116ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe716563700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe70000f010 tx=0x7fe7000047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b013d9f71007e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7000325b0 con 0x559d911030e0 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe70001d450 con 0x559d911030e0 2023-10-08T09:55:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe70001d910 con 0x559d911030e0 2023-10-08T09:55:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70001d610 con 0x559d911030e0 2023-10-08T09:55:57.529 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:55:57.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:57 smithi186 ceph-mon[104393]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2023-10-08T09:55:57.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2482690704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1654920408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/4002641518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 10 client.15711 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d90e80e60 con 0x559d911030e0 2023-10-08T09:55:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe70001d910 con 0x559d911030e0 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe700040020 con 0x559d911030e0 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 20 client.15711 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d9116ec80 0x559d91171070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 4 client.15711 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.517+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559d9104ebb0 con 0x559d9116ec80 2023-10-08T09:55:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.518+0000 7fe715d62700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d9116ec80 0x559d91171070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.518+0000 7fe6f9ffb700 10 client.15711 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.518+0000 7fe715d62700 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d9116ec80 0x559d91171070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa31f363e20df2 server_cookie=b73c8398eed9a85 in_seq=0 out_seq=0 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe6f9ffb700 1 -- 172.21.15.97:0/3302228849 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559d9104ebb0 con 0x559d9116ec80 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe6f9ffb700 10 client.15711 handle_command_reply: tid=0 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 client.15711 shutdown 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 msgr2=0x559d911136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe70000f010 tx=0x7fe7000047a0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 shutdown_connections 2023-10-08T09:55:57.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559d9116ec80 0x559d91171070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559d91104880 0x559d9116ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe6f9ffb700 0 client.15711 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d911030e0 0x559d911136e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 --2- 172.21.15.97:0/3302228849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559d91103a40 0x559d91113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 >> 172.21.15.97:0/3302228849 conn(0x559d910772e0 msgr2=0x559d9110ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 shutdown_connections 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 1 -- 172.21.15.97:0/3302228849 wait complete. 2023-10-08T09:55:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7249b0b80 20 client.15711 trim_cache size 0 max 16384 2023-10-08T09:55:57.536 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:57.536 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2023-10-08T09:55:57.536 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "3a33ece6-63ba-4124-9ab1-ac366e4d0b35", 2023-10-08T09:55:57.536 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2023-10-08T09:55:57.536 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:57.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 msgr2=0x7fe6fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 0x7fe6fc0a3940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe710101810 tx=0x7fe70000a680 comp rx=0 tx=0).stop 2023-10-08T09:55:57.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 msgr2=0x7fe710198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00dd10 comp rx=0 tx=0).stop 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 shutdown_connections 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe6fc0a1480 0x7fe6fc0a3940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.519+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe710102350 0x7fe71019ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.520+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7101007c0 0x7fe710198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.520+0000 7fe7187c7700 1 --2- 172.21.15.97:0/1744138807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7101019b0 0x7fe710198f60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.520+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 >> 172.21.15.97:0/1744138807 conn(0x7fe7100fbd30 msgr2=0x7fe710072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.520+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 shutdown_connections 2023-10-08T09:55:57.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.520+0000 7fe7187c7700 1 -- 172.21.15.97:0/1744138807 wait complete. 2023-10-08T09:55:57.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:55:57.594 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-08T09:55:57.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 -- 172.21.15.97:0/736585166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 msgr2=0x7ff890100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 --2- 172.21.15.97:0/736585166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890100d20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff8800099c0 tx=0x7ff880009cd0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 -- 172.21.15.97:0/736585166 shutdown_connections 2023-10-08T09:55:57.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 --2- 172.21.15.97:0/736585166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 0x7ff890107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 --2- 172.21.15.97:0/736585166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff8900ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 --2- 172.21.15.97:0/736585166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890100d20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 -- 172.21.15.97:0/736585166 >> 172.21.15.97:0/736585166 conn(0x7ff8900fac40 msgr2=0x7ff8900fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 -- 172.21.15.97:0/736585166 shutdown_connections 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 -- 172.21.15.97:0/736585166 wait complete. 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.701+0000 7ff895669700 1 Processor -- start 2023-10-08T09:55:57.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 -- start start 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff890196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 0x7ff89019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019e260 con 0x7ff8901008c0 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019e3a0 con 0x7ff890101260 2023-10-08T09:55:57.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019e530 con 0x7ff8900ff6d0 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57210/0 (socket says 172.21.15.97:57210) 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 -- 172.21.15.97:0/2412609442 learned_addr learned my addr 172.21.15.97:0/2412609442 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 0x7ff89019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88effd700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff890196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 -- 172.21.15.97:0/2412609442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 msgr2=0x7ff890196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff890196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 -- 172.21.15.97:0/2412609442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 msgr2=0x7ff89019dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 0x7ff89019dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 -- 172.21.15.97:0/2412609442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800095f0 con 0x7ff8901008c0 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88effd700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff890196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:55:57.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff8800036f0 tx=0x7ff8800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff877fff700 1 -- 172.21.15.97:0/2412609442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88001d070 con 0x7ff8901008c0 2023-10-08T09:55:57.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff877fff700 1 -- 172.21.15.97:0/2412609442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff880004090 con 0x7ff8901008c0 2023-10-08T09:55:57.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.702+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89019e7b0 con 0x7ff8901008c0 2023-10-08T09:55:57.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.703+0000 7ff877fff700 1 -- 172.21.15.97:0/2412609442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880022d40 con 0x7ff8901008c0 2023-10-08T09:55:57.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.703+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89019ec60 con 0x7ff8901008c0 2023-10-08T09:55:57.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:57 smithi178 ceph-mon[103156]: pgmap v324: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s 2023-10-08T09:55:57.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2482690704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1654920408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:55:57.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/4002641518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:55:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.705+0000 7ff877fff700 1 -- 172.21.15.97:0/2412609442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff880022470 con 0x7ff8901008c0 2023-10-08T09:55:57.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.705+0000 7ff877fff700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 0x7ff8780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.706+0000 7ff88effd700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 0x7ff8780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.706+0000 7ff877fff700 1 -- 172.21.15.97:0/2412609442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff8800ce1b0 con 0x7ff8901008c0 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.706+0000 7ff88effd700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 0x7ff8780a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff884005fd0 tx=0x7ff884005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.706+0000 7ff8a1852b80 1 Processor -- start 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 -- start start 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee89c0 0x561f59ee9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee9800 0x561f59ee9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59e50460 con 0x561f59ee8060 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59e41c20 con 0x561f59ee9800 2023-10-08T09:55:57.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59e3f540 con 0x561f59ee89c0 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57216/0 (socket says 172.21.15.97:57216) 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 -- 172.21.15.97:0/2644192644 learned_addr learned my addr 172.21.15.97:0/2644192644 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 -- 172.21.15.97:0/2644192644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee89c0 msgr2=0x561f59ee9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee89c0 0x561f59ee9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee9800 0x561f59ee9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 -- 172.21.15.97:0/2644192644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee9800 msgr2=0x561f59ee9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee9800 0x561f59ee9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 -- 172.21.15.97:0/2644192644 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f59c65de0 con 0x561f59ee8060 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff88effd700 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff88400db90 tx=0x7ff884010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8456a9209addf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.707+0000 7ff8757fa700 1 -- 172.21.15.97:0/2644192644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88400f930 con 0x561f59ee8060 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8757fa700 1 -- 172.21.15.97:0/2644192644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88401c450 con 0x561f59ee8060 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8757fa700 1 -- 172.21.15.97:0/2644192644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88401b610 con 0x561f59ee8060 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 -- 172.21.15.97:0/2644192644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 msgr2=0x561f59ee8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff88400db90 tx=0x7ff884010000 comp rx=0 tx=0).stop 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 -- 172.21.15.97:0/2644192644 shutdown_connections 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee9800 0x561f59ee9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee8060 0x561f59ee8480 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/2644192644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee89c0 0x561f59ee9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 -- 172.21.15.97:0/2644192644 >> 172.21.15.97:0/2644192644 conn(0x561f59e5c260 msgr2=0x561f59ee2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 -- 172.21.15.97:0/2644192644 shutdown_connections 2023-10-08T09:55:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.708+0000 7ff8a1852b80 1 -- 172.21.15.97:0/2644192644 wait complete. 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 -- start start 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 Processor -- start 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59e3ae30 con 0x561f59ee89c0 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59dd7710 con 0x561f59ee8060 2023-10-08T09:55:57.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59dd73b0 con 0x561f59ee9800 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff8a1852b80 10 client.0 _ll_register_callbacks cb 0x561f59c34800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57224/0 (socket says 172.21.15.97:57224) 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3635551326 learned_addr learned my addr 172.21.15.97:0/3635551326 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88effd700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3635551326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 msgr2=0x561f59ef9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 msgr2=0x561f59ef1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3635551326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88400d840 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c010 tx=0x7ff88001af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf0a82a399eba19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.709+0000 7ff872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880022ea0 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff8800cbe40 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f59e02eb0 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff88009f160 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 10 client.15723 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561f59c4ec60 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f59e02eb0 con 0x561f59ee89c0 2023-10-08T09:55:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff8800a30a0 con 0x561f59ee89c0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 20 client.15723 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59efd8a0 0x561f59effcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 4 client.15723 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f59e33e10 con 0x561f59efd8a0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.710+0000 7ff88effd700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59efd8a0 0x561f59effcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.711+0000 7ff872ff5700 10 client.15723 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.711+0000 7ff88effd700 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59efd8a0 0x561f59effcc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2c6d3b54bf6314 server_cookie=29c8f6536a249ccc in_seq=0 out_seq=0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.711+0000 7ff872ff5700 1 -- 172.21.15.97:0/3635551326 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f59e33e10 con 0x561f59efd8a0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.711+0000 7ff872ff5700 10 client.15723 handle_command_reply: tid=0 2023-10-08T09:55:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.711+0000 7ff8a1852b80 1 client.15723 shutdown 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 msgr2=0x561f59ef2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c010 tx=0x7ff88001af80 comp rx=0 tx=0).stop 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 shutdown_connections 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59efd8a0 0x561f59effcc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3635551326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef2440 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff872ff5700 0 client.15723 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 >> 172.21.15.97:0/3635551326 conn(0x561f59e5c260 msgr2=0x561f59ee24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 shutdown_connections 2023-10-08T09:55:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3635551326 wait complete. 2023-10-08T09:55:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.712+0000 7ff8a1852b80 20 client.15723 trim_cache size 0 max 16384 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 Processor -- start 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 -- start start 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ee2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59edef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59cbb920 con 0x561f59ee89c0 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59e02ee0 con 0x561f59ee8060 2023-10-08T09:55:57.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59ee4910 con 0x561f59ee9800 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57232/0 (socket says 172.21.15.97:57232) 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff88e7fc700 1 -- 172.21.15.97:0/4185714343 learned_addr learned my addr 172.21.15.97:0/4185714343 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.729+0000 7ff88effd700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ee2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 -- 172.21.15.97:0/4185714343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 msgr2=0x561f59edef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59edef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59edef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 -- 172.21.15.97:0/4185714343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 msgr2=0x561f59ee2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ee2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 -- 172.21.15.97:0/4185714343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f59c8c250 con 0x561f59ee89c0 2023-10-08T09:55:57.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88effd700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ee2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff880097c50 tx=0x7ff880009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5823657f99ebdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff875ffb700 1 -- 172.21.15.97:0/4185714343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880022ea0 con 0x561f59ee89c0 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff875ffb700 1 -- 172.21.15.97:0/4185714343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8800a1190 con 0x561f59ee89c0 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff875ffb700 1 -- 172.21.15.97:0/4185714343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88009f410 con 0x561f59ee89c0 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff8a1852b80 1 -- 172.21.15.97:0/4185714343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 msgr2=0x561f59ef93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.730+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff880097c50 tx=0x7ff880009310 comp rx=0 tx=0).stop 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- 172.21.15.97:0/4185714343 shutdown_connections 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59edef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ee2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/4185714343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef93f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- 172.21.15.97:0/4185714343 >> 172.21.15.97:0/4185714343 conn(0x561f59e5c260 msgr2=0x561f59f04620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- 172.21.15.97:0/4185714343 shutdown_connections 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- 172.21.15.97:0/4185714343 wait complete. 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- start start 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 Processor -- start 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59ef8f60 con 0x561f59ee89c0 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59ef90d0 con 0x561f59ee8060 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f59ef9290 con 0x561f59ee9800 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff8a1852b80 10 client.0 _ll_register_callbacks cb 0x7ff880097660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:55:57.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57238/0 (socket says 172.21.15.97:57238) 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3700964872 learned_addr learned my addr 172.21.15.97:0/3700964872 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88effd700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3700964872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 msgr2=0x561f59ef8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.731+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 msgr2=0x561f59ef7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:55:57.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff88e7fc700 1 -- 172.21.15.97:0/3700964872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f59c4ec60 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff88f7fe700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff88e7fc700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff88009b1c0 tx=0x7ff8800037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96c873d02971034 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880022ea0 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff8800cbe40 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff88401c450 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff88009f3f0 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 10 client.15729 fetch_fsmap learned FSMap version 24 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561f59c642b0 con 0x561f59ee89c0 2023-10-08T09:55:57.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88401c450 con 0x561f59ee89c0 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff8800970b0 con 0x561f59ee89c0 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 20 client.15729 populate_metadata read hostname 'smithi097' 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59f52350 0x561f59f54740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 4 client.15729 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.732+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f59e33b30 con 0x561f59f52350 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff88effd700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59f52350 0x561f59f54740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff88effd700 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59f52350 0x561f59f54740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf515331eb1773e3 server_cookie=56d3c21a758b0ad2 in_seq=0 out_seq=0 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff872ff5700 10 client.15729 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff872ff5700 1 -- 172.21.15.97:0/3700964872 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561f59e33b30 con 0x561f59f52350 2023-10-08T09:55:57.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff872ff5700 10 client.15729 handle_command_reply: tid=0 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 client.15729 shutdown 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 msgr2=0x561f59ef7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff88009b1c0 tx=0x7ff8800037c0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 shutdown_connections 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561f59f52350 0x561f59f54740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f59ee9800 0x561f59ef8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff872ff5700 0 client.15729 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f59ee8060 0x561f59ef7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 --2- 172.21.15.97:0/3700964872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561f59ee89c0 0x561f59ef7ab0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 >> 172.21.15.97:0/3700964872 conn(0x561f59e5c260 msgr2=0x561f59edef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 shutdown_connections 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.733+0000 7ff8a1852b80 1 -- 172.21.15.97:0/3700964872 wait complete. 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff8a1852b80 20 client.15729 trim_cache size 0 max 16384 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "3a33ece6-63ba-4124-9ab1-ac366e4d0b35": { 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2023-10-08T09:55:57.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "3a33ece6-63ba-4124-9ab1-ac366e4d0b35", 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 msgr2=0x7ff8780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 0x7ff8780a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff884005fd0 tx=0x7ff884005dc0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 msgr2=0x7ff890196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff8800036f0 tx=0x7ff8800047b0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 shutdown_connections 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff8780a1480 0x7ff8780a3940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff890101260 0x7ff89019dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8900ff6d0 0x7ff890196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 --2- 172.21.15.97:0/2412609442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8901008c0 0x7ff890196c70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 >> 172.21.15.97:0/2412609442 conn(0x7ff8900fac40 msgr2=0x7ff8900fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 shutdown_connections 2023-10-08T09:55:57.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:55:57.734+0000 7ff895669700 1 -- 172.21.15.97:0/2412609442 wait complete. 2023-10-08T09:55:57.748 INFO:tasks.cephfs.filesystem:scrub status for tag:3a33ece6-63ba-4124-9ab1-ac366e4d0b35 - {'path': '/', 'tag': '3a33ece6-63ba-4124-9ab1-ac366e4d0b35', 'options': 'recursive,force'} 2023-10-08T09:55:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2968314497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:58 smithi186 ceph-mon[104393]: scrub queued for path: / 2023-10-08T09:55:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:58 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:58 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:55:58.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:58 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:58.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2968314497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:58 smithi097 ceph-mon[99491]: scrub queued for path: / 2023-10-08T09:55:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:58 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:58 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:55:58.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:58 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2968314497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:55:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:58 smithi178 ceph-mon[103156]: scrub queued for path: / 2023-10-08T09:55:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:58 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:58 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:55:58.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:58 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:55:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:55:59 smithi186 ceph-mon[104393]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-08T09:55:59.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:55:59 smithi097 ceph-mon[99491]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-08T09:55:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:55:59 smithi178 ceph-mon[103156]: pgmap v325: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-08T09:56:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:01 smithi186 ceph-mon[104393]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:56:01.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:01 smithi097 ceph-mon[99491]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:56:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:01 smithi178 ceph-mon[103156]: pgmap v326: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T09:56:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:03 smithi186 ceph-mon[104393]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:56:03.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:03 smithi097 ceph-mon[99491]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:56:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:03 smithi178 ceph-mon[103156]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-08T09:56:05.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:05 smithi186 ceph-mon[104393]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:05.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:05 smithi097 ceph-mon[99491]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:05.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:05 smithi178 ceph-mon[103156]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:07.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:07 smithi186 ceph-mon[104393]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:07.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:07 smithi097 ceph-mon[99491]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:07.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:07 smithi178 ceph-mon[103156]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:08.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:08 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:56:08.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:08 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:56:08.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:08 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:56:09.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:09 smithi186 ceph-mon[104393]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:09.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:09 smithi097 ceph-mon[99491]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:09.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:09 smithi178 ceph-mon[103156]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/2315577290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 msgr2=0x7f4ef0100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/2315577290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef0100450 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80099c0 tx=0x7f4ed8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/2315577290 shutdown_connections 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/2315577290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 0x7f4ef0107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/2315577290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 0x7f4ef00ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/2315577290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef0100450 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/2315577290 >> 172.21.15.97:0/2315577290 conn(0x7f4ef00fa370 msgr2=0x7f4ef00fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/2315577290 shutdown_connections 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/2315577290 wait complete. 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 Processor -- start 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.521+0000 7f4ef5c7b700 1 -- start start 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 0x7f4ef010bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 0x7f4ef019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019e100 con 0x7f4ef00ffff0 2023-10-08T09:56:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019e270 con 0x7f4ef00fee00 2023-10-08T09:56:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019e430 con 0x7f4ef0100990 2023-10-08T09:56:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57252/0 (socket says 172.21.15.97:57252) 2023-10-08T09:56:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 -- 172.21.15.97:0/749291618 learned_addr learned my addr 172.21.15.97:0/749291618 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeffff700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 0x7f4ef019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 0x7f4ef010bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 -- 172.21.15.97:0/749291618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 msgr2=0x7f4ef019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 0x7f4ef019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 -- 172.21.15.97:0/749291618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 msgr2=0x7f4ef010bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 0x7f4ef010bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 -- 172.21.15.97:0/749291618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80095f0 con 0x7f4ef00ffff0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009fd0 tx=0x7f4ed8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eecff9700 1 -- 172.21.15.97:0/749291618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed801d070 con 0x7f4ef00ffff0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eecff9700 1 -- 172.21.15.97:0/749291618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed80059a0 con 0x7f4ef00ffff0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef019e650 con 0x7f4ef00ffff0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4eecff9700 1 -- 172.21.15.97:0/749291618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80173f0 con 0x7f4ef00ffff0 2023-10-08T09:56:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.522+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef019eb00 con 0x7f4ef00ffff0 2023-10-08T09:56:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.524+0000 7f4eecff9700 1 -- 172.21.15.97:0/749291618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4ed8022670 con 0x7f4ef00ffff0 2023-10-08T09:56:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.525+0000 7f4eecff9700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 0x7f4edc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.525+0000 7f4eecff9700 1 -- 172.21.15.97:0/749291618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ed80ce130 con 0x7f4ef00ffff0 2023-10-08T09:56:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.525+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 0x7f4edc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 0x7f4edc0a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ffe50 tx=0x7f4ee800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 Processor -- start 2023-10-08T09:56:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 -- start start 2023-10-08T09:56:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf71b400 con 0x55dfcf7b42b0 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf70cbc0 con 0x55dfcf7b2e40 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf70a4e0 con 0x55dfcf7b37a0 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57262/0 (socket says 172.21.15.97:57262) 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eeffff700 1 -- 172.21.15.97:0/2236736344 learned_addr learned my addr 172.21.15.97:0/2236736344 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.526+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 -- 172.21.15.97:0/2236736344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 msgr2=0x55dfcf7b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 -- 172.21.15.97:0/2236736344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 msgr2=0x55dfcf7b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 -- 172.21.15.97:0/2236736344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcf530d80 con 0x55dfcf7b42b0 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4eeffff700 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0011680 tx=0x7f4ee0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb51aa018fc5098 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4ee5ffb700 1 -- 172.21.15.97:0/2236736344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000e040 con 0x55dfcf7b42b0 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4ee5ffb700 1 -- 172.21.15.97:0/2236736344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee000f040 con 0x55dfcf7b42b0 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4ee5ffb700 1 -- 172.21.15.97:0/2236736344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0004860 con 0x55dfcf7b42b0 2023-10-08T09:56:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4f01e64b80 1 -- 172.21.15.97:0/2236736344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 msgr2=0x55dfcf7b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.527+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0011680 tx=0x7f4ee0009920 comp rx=0 tx=0).stop 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- 172.21.15.97:0/2236736344 shutdown_connections 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7b4730 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/2236736344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- 172.21.15.97:0/2236736344 >> 172.21.15.97:0/2236736344 conn(0x55dfcf727200 msgr2=0x55dfcf7ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- 172.21.15.97:0/2236736344 shutdown_connections 2023-10-08T09:56:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- 172.21.15.97:0/2236736344 wait complete. 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- start start 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 Processor -- start 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf705dd0 con 0x55dfcf7b2e40 2023-10-08T09:56:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf6a26b0 con 0x55dfcf7b42b0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf6a2350 con 0x55dfcf7b37a0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4f01e64b80 10 client.0 _ll_register_callbacks cb 0x55dfcf4ff7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57264/0 (socket says 172.21.15.97:57264) 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.528+0000 7f4eef7fe700 1 -- 172.21.15.97:0/3353944540 learned_addr learned my addr 172.21.15.97:0/3353944540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eeffff700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 -- 172.21.15.97:0/3353944540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 msgr2=0x55dfcf7bcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 -- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 msgr2=0x55dfcf7c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 -- 172.21.15.97:0/3353944540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee0011330 con 0x55dfcf7b2e40 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eeffff700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ee801da60 tx=0x7f4ee80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266cfbdce064964 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8010840 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ee80109a0 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfcf6cde50 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee8010b60 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4f01e64b80 10 client.15741 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dfcf519c00 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.529+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfcf6cde50 con 0x55dfcf7b2e40 2023-10-08T09:56:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4ee801b420 con 0x55dfcf7b2e40 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 20 client.15741 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf7c86d0 0x55dfcf7caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 4 client.15741 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfcf6fedb0 con 0x55dfcf7c86d0 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf7c86d0 0x55dfcf7caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4eceffd700 10 client.15741 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.530+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf7c86d0 0x55dfcf7caaf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0be5a6aff64fc7 server_cookie=53e599babe0a7db6 in_seq=0 out_seq=0 2023-10-08T09:56:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4eceffd700 1 -- 172.21.15.97:0/3353944540 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dfcf6fedb0 con 0x55dfcf7c86d0 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4eceffd700 10 client.15741 handle_command_reply: tid=0 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 client.15741 shutdown 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 msgr2=0x55dfcf7bc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ee801da60 tx=0x7f4ee80047c0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 shutdown_connections 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc890 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf7c86d0 0x55dfcf7caaf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3353944540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 >> 172.21.15.97:0/3353944540 conn(0x55dfcf727200 msgr2=0x55dfcf7ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4eceffd700 0 client.15741 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 shutdown_connections 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3353944540 wait complete. 2023-10-08T09:56:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.531+0000 7f4f01e64b80 20 client.15741 trim_cache size 0 max 16384 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 Processor -- start 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 -- start start 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf5868c0 con 0x55dfcf7b37a0 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf7c4f20 con 0x55dfcf7b2e40 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf7af300 con 0x55dfcf7b42b0 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57274/0 (socket says 172.21.15.97:57274) 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 -- 172.21.15.97:0/3230151581 learned_addr learned my addr 172.21.15.97:0/3230151581 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 -- 172.21.15.97:0/3230151581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b42b0 msgr2=0x55dfcf7bcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 -- 172.21.15.97:0/3230151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 msgr2=0x55dfcf7bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 -- 172.21.15.97:0/3230151581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcf5571f0 con 0x55dfcf7b37a0 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.546+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8097c50 tx=0x7f4ed80cb800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7080ef36803081 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4ee67fc700 1 -- 172.21.15.97:0/3230151581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed809b1e0 con 0x55dfcf7b37a0 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4ee67fc700 1 -- 172.21.15.97:0/3230151581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed80b2070 con 0x55dfcf7b37a0 2023-10-08T09:56:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4ee67fc700 1 -- 172.21.15.97:0/3230151581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80b9270 con 0x55dfcf7b37a0 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3230151581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 msgr2=0x55dfcf7c3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8097c50 tx=0x7f4ed80cb800 comp rx=0 tx=0).stop 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3230151581 shutdown_connections 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/3230151581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7c3d30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3230151581 >> 172.21.15.97:0/3230151581 conn(0x55dfcf727200 msgr2=0x55dfcf7b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3230151581 shutdown_connections 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- 172.21.15.97:0/3230151581 wait complete. 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.547+0000 7f4f01e64b80 1 -- start start 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 Processor -- start 2023-10-08T09:56:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf7c32c0 con 0x55dfcf7b2e40 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf7c3430 con 0x55dfcf7b42b0 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcf7c35f0 con 0x55dfcf7b37a0 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4f01e64b80 10 client.0 _ll_register_callbacks cb 0x7f4ee801d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57278/0 (socket says 172.21.15.97:57278) 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 -- 172.21.15.97:0/1741746069 learned_addr learned my addr 172.21.15.97:0/1741746069 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 -- 172.21.15.97:0/1741746069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 msgr2=0x55dfcf7bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 -- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 msgr2=0x55dfcf7c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eeffff700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 -- 172.21.15.97:0/1741746069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef019eb00 con 0x55dfcf7b2e40 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eef7fe700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ec0 tx=0x7f4ee800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e72c33e87317e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.548+0000 7f4eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8010640 con 0x55dfcf7b2e40 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ee80107a0 con 0x55dfcf7b2e40 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee000f040 con 0x55dfcf7b2e40 2023-10-08T09:56:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee8010960 con 0x55dfcf7b2e40 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 10 client.15747 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dfcf52f250 con 0x55dfcf7b2e40 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee000f040 con 0x55dfcf7b2e40 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4ee8036b90 con 0x55dfcf7b2e40 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 20 client.15747 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf81d2f0 0x55dfcf81f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 4 client.15747 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfcf6fead0 con 0x55dfcf81d2f0 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.549+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf81d2f0 0x55dfcf81f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.550+0000 7f4eceffd700 10 client.15747 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.550+0000 7f4eeeffd700 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf81d2f0 0x55dfcf81f730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff88dca8e67e1c1a server_cookie=a669e40aca50026c in_seq=0 out_seq=0 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4eceffd700 1 -- 172.21.15.97:0/1741746069 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfcf6fead0 con 0x55dfcf81d2f0 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4eceffd700 10 client.15747 handle_command_reply: tid=0 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 client.15747 shutdown 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 msgr2=0x55dfcf7bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ec0 tx=0x7f4ee800a620 comp rx=0 tx=0).stop 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 shutdown_connections 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfcf7b42b0 0x55dfcf7c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfcf7b2e40 0x55dfcf7bf870 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dfcf81d2f0 0x55dfcf81f730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 --2- 172.21.15.97:0/1741746069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfcf7b37a0 0x55dfcf7bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4eceffd700 0 client.15747 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 >> 172.21.15.97:0/1741746069 conn(0x55dfcf727200 msgr2=0x55dfcf7bc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 shutdown_connections 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 1 -- 172.21.15.97:0/1741746069 wait complete. 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.556+0000 7f4f01e64b80 20 client.15747 trim_cache size 0 max 16384 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:56:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:56:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:56:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 msgr2=0x7f4edc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 0x7f4edc0a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ffe50 tx=0x7f4ee800a3b0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 msgr2=0x7f4ef019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009fd0 tx=0x7f4ed8005f10 comp rx=0 tx=0).stop 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 shutdown_connections 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4edc0a1480 0x7f4edc0a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ef0100990 0x7f4ef019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ef00fee00 0x7f4ef010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 --2- 172.21.15.97:0/749291618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ef00ffff0 0x7f4ef019b2f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 >> 172.21.15.97:0/749291618 conn(0x7f4ef00fa370 msgr2=0x7f4ef00fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 shutdown_connections 2023-10-08T09:56:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:10.557+0000 7f4ef5c7b700 1 -- 172.21.15.97:0/749291618 wait complete. 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:11 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:11 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:11 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T09:56:12.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:56:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:56:12.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:56:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:56:12.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:56:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:56:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:56:13.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:13 smithi097 ceph-mon[99491]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-10-08T09:56:13.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:13 smithi178 ceph-mon[103156]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-10-08T09:56:13.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:13 smithi186 ceph-mon[104393]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-10-08T09:56:15.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:15 smithi097 ceph-mon[99491]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-08T09:56:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:15 smithi178 ceph-mon[103156]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-08T09:56:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:15 smithi186 ceph-mon[104393]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-08T09:56:17.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:17 smithi097 ceph-mon[99491]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:17.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:17 smithi178 ceph-mon[103156]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:17 smithi186 ceph-mon[104393]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:19.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:19 smithi097 ceph-mon[99491]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:19 smithi178 ceph-mon[103156]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:19.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:19 smithi186 ceph-mon[104393]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:21.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:21 smithi097 ceph-mon[99491]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-08T09:56:21.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:21 smithi178 ceph-mon[103156]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-08T09:56:21.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:21 smithi186 ceph-mon[104393]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-08T09:56:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:23 smithi097 ceph-mon[99491]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-08T09:56:23.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:23 smithi178 ceph-mon[103156]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-08T09:56:23.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:23 smithi186 ceph-mon[104393]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-08T09:56:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/4079468288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 msgr2=0x7f9b9c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/4079468288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c0ffa60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0099c0 tx=0x7f9b8c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/4079468288 shutdown_connections 2023-10-08T09:56:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/4079468288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 0x7f9b9c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/4079468288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c0ffa60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/4079468288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/4079468288 >> 172.21.15.97:0/4079468288 conn(0x7f9b9c0fabb0 msgr2=0x7f9b9c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/4079468288 shutdown_connections 2023-10-08T09:56:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.671+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/4079468288 wait complete. 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 Processor -- start 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 -- start start 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 0x7f9b9c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19e1a0 con 0x7f9b9c0ff640 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19e310 con 0x7f9b9c100830 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9ba0d4c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19e4d0 con 0x7f9b9c1011d0 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57290/0 (socket says 172.21.15.97:57290) 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9b9a59c700 1 -- 172.21.15.97:0/105173984 learned_addr learned my addr 172.21.15.97:0/105173984 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.672+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 0x7f9b9c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 -- 172.21.15.97:0/105173984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 msgr2=0x7f9b9c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 0x7f9b9c19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 -- 172.21.15.97:0/105173984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 msgr2=0x7f9b9c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 -- 172.21.15.97:0/105173984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0095f0 con 0x7f9b9c0ff640 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c006010 tx=0x7f9b8c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b8b7fe700 1 -- 172.21.15.97:0/105173984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c01d070 con 0x7f9b9c0ff640 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b8b7fe700 1 -- 172.21.15.97:0/105173984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c004150 con 0x7f9b9c0ff640 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c19e6f0 con 0x7f9b9c0ff640 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9b8b7fe700 1 -- 172.21.15.97:0/105173984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c021410 con 0x7f9b9c0ff640 2023-10-08T09:56:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.673+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c19eb10 con 0x7f9b9c0ff640 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.676+0000 7f9b8b7fe700 1 -- 172.21.15.97:0/105173984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9b8c00f6f0 con 0x7f9b9c0ff640 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.677+0000 7f9b8b7fe700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 0x7f9b840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.677+0000 7f9b8b7fe700 1 -- 172.21.15.97:0/105173984 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b8c0d1170 con 0x7f9b9c0ff640 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.677+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 0x7f9b840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.677+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 0x7f9b840a3940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 Processor -- start 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 -- start start 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b63f400 con 0x55fd4b6d6e40 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b630bc0 con 0x55fd4b6d82b0 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b62e4e0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57298/0 (socket says 172.21.15.97:57298) 2023-10-08T09:56:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 -- 172.21.15.97:0/3842669651 learned_addr learned my addr 172.21.15.97:0/3842669651 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 -- 172.21.15.97:0/3842669651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 msgr2=0x55fd4b6d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 -- 172.21.15.97:0/3842669651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d82b0 msgr2=0x55fd4b6d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.678+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6d8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b9a59c700 1 -- 172.21.15.97:0/3842669651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4b454d80 con 0x55fd4b6d6e40 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005f50 tx=0x7f9b8c00fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4bef72b57bcf57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b88ff9700 1 -- 172.21.15.97:0/3842669651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c09a900 con 0x55fd4b6d6e40 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b88ff9700 1 -- 172.21.15.97:0/3842669651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c09a720 con 0x55fd4b6d6e40 2023-10-08T09:56:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.679+0000 7f9b88ff9700 1 -- 172.21.15.97:0/3842669651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c099160 con 0x55fd4b6d6e40 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3842669651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 msgr2=0x55fd4b6d7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005f50 tx=0x7f9b8c00fa90 comp rx=0 tx=0).stop 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3842669651 shutdown_connections 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d7260 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3842669651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3842669651 >> 172.21.15.97:0/3842669651 conn(0x55fd4b64b200 msgr2=0x55fd4b6d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3842669651 shutdown_connections 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3842669651 wait complete. 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- start start 2023-10-08T09:56:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 Processor -- start 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b629dd0 con 0x55fd4b6d6e40 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b5c66b0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b5c6350 con 0x55fd4b6d82b0 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9bacf35b80 10 client.0 _ll_register_callbacks cb 0x55fd4b4237a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.680+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57304/0 (socket says 172.21.15.97:57304) 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 -- 172.21.15.97:0/952824817 learned_addr learned my addr 172.21.15.97:0/952824817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 -- 172.21.15.97:0/952824817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 msgr2=0x55fd4b6e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 -- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 msgr2=0x55fd4b6e09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 -- 172.21.15.97:0/952824817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c19eb10 con 0x55fd4b6d6e40 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a10c0 tx=0x7f9b8c0cb4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb7962eb2ee9978 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c09adc0 con 0x55fd4b6d6e40 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b8c09b4c0 con 0x55fd4b6d6e40 2023-10-08T09:56:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4b5f1e50 con 0x55fd4b6d6e40 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8c0a2c30 con 0x55fd4b6d6e40 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9bacf35b80 10 client.15759 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fd4b43dc00 con 0x55fd4b6d6e40 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.681+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c066020 con 0x55fd4b6d6e40 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b8c0cdb80 con 0x55fd4b6d6e40 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 20 client.15759 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b6ebdd0 0x55fd4b6ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 4 client.15759 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd4b622db0 con 0x55fd4b6ebdd0 2023-10-08T09:56:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.682+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b6ebdd0 0x55fd4b6ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b6ebdd0 0x55fd4b6ee1f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e37e4d7166e44e server_cookie=2917c5718ae8d4c2 in_seq=0 out_seq=0 2023-10-08T09:56:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9b81ffb700 10 client.15759 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9b81ffb700 1 -- 172.21.15.97:0/952824817 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd4b622db0 con 0x55fd4b6ebdd0 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9b81ffb700 10 client.15759 handle_command_reply: tid=0 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 client.15759 shutdown 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 msgr2=0x55fd4b6e04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a10c0 tx=0x7f9b8c0cb4b0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 shutdown_connections 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b6ebdd0 0x55fd4b6ee1f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e04b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9b81ffb700 0 client.15759 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/952824817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 >> 172.21.15.97:0/952824817 conn(0x55fd4b64b200 msgr2=0x55fd4b6d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.683+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 shutdown_connections 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.684+0000 7f9bacf35b80 1 -- 172.21.15.97:0/952824817 wait complete. 2023-10-08T09:56:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.684+0000 7f9bacf35b80 20 client.15759 trim_cache size 0 max 16384 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 Processor -- start 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 -- start start 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b4aa8c0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b6ce830 con 0x55fd4b6d6e40 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b6cc2b0 con 0x55fd4b6d82b0 2023-10-08T09:56:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57314/0 (socket says 172.21.15.97:57314) 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 -- 172.21.15.97:0/3388498576 learned_addr learned my addr 172.21.15.97:0/3388498576 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 -- 172.21.15.97:0/3388498576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 msgr2=0x55fd4b6de7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6de7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 -- 172.21.15.97:0/3388498576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 msgr2=0x55fd4b6d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b99d9b700 1 -- 172.21.15.97:0/3388498576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4b47b1f0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.698+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c068ba0 tx=0x7f9b90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f7d7ea61231c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9b897fa700 1 -- 172.21.15.97:0/3388498576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90022040 con 0x55fd4b6d77a0 2023-10-08T09:56:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9b897fa700 1 -- 172.21.15.97:0/3388498576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9000a960 con 0x55fd4b6d77a0 2023-10-08T09:56:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9b897fa700 1 -- 172.21.15.97:0/3388498576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9001d680 con 0x55fd4b6d77a0 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3388498576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 msgr2=0x55fd4b6e79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.699+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c068ba0 tx=0x7f9b90014000 comp rx=0 tx=0).stop 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3388498576 shutdown_connections 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/3388498576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e79a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3388498576 >> 172.21.15.97:0/3388498576 conn(0x55fd4b64b200 msgr2=0x55fd4b6dfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3388498576 shutdown_connections 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- 172.21.15.97:0/3388498576 wait complete. 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- start start 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 Processor -- start 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b6e6030 con 0x55fd4b6d77a0 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b6e6170 con 0x55fd4b6d6e40 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4b6e6330 con 0x55fd4b6d82b0 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9bacf35b80 10 client.0 _ll_register_callbacks cb 0x55fd4b4237a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9b99d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9b99d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57320/0 (socket says 172.21.15.97:57320) 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9b99d9b700 1 -- 172.21.15.97:0/4266307055 learned_addr learned my addr 172.21.15.97:0/4266307055 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.700+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b9ad9d700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 -- 172.21.15.97:0/4266307055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 msgr2=0x55fd4b6e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 -- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 msgr2=0x55fd4b6e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 -- 172.21.15.97:0/4266307055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9000ede0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b99d9b700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9b90010cc0 tx=0x7f9b90025660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ed4a10d97f2791 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90010840 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b90004850 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4b622ad0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b9000a960 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9bacf35b80 10 client.15765 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fd4b453250 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.701+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd4b622ad0 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b90025050 con 0x55fd4b6d77a0 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 20 client.15765 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b740f30 0x55fd4b743350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 4 client.15765 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:56:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd4b475f10 con 0x55fd4b740f30 2023-10-08T09:56:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b740f30 0x55fd4b743350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9b81ffb700 10 client.15765 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.702+0000 7f9b9a59c700 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b740f30 0x55fd4b743350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aee59bf80d3af9a server_cookie=41b5979c6b13f117 in_seq=0 out_seq=0 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9b81ffb700 1 -- 172.21.15.97:0/4266307055 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd4b475f10 con 0x55fd4b740f30 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9b81ffb700 10 client.15765 handle_command_reply: tid=0 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 client.15765 shutdown 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 msgr2=0x55fd4b6e7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9b90010cc0 tx=0x7f9b90025660 comp rx=0 tx=0).stop 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 shutdown_connections 2023-10-08T09:56:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55fd4b740f30 0x55fd4b743350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fd4b6d82b0 0x55fd4b6e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9b81ffb700 0 client.15765 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd4b6d6e40 0x55fd4b6e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 --2- 172.21.15.97:0/4266307055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd4b6d77a0 0x55fd4b6e7830 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 >> 172.21.15.97:0/4266307055 conn(0x55fd4b64b200 msgr2=0x55fd4b6e0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 shutdown_connections 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 1 -- 172.21.15.97:0/4266307055 wait complete. 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9bacf35b80 20 client.15765 trim_cache size 0 max 16384 2023-10-08T09:56:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:56:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:56:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:56:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:56:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 msgr2=0x7f9b840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.706+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 0x7f9b840a3940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b90005fd0 tx=0x7f9b90005e30 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 msgr2=0x7f9b9c068600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c006010 tx=0x7f9b8c005af0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 shutdown_connections 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9b840a1480 0x7f9b840a3940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b9c1011d0 0x7f9b9c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0ff640 0x7f9b9c068600 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 --2- 172.21.15.97:0/105173984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b9c100830 0x7f9b9c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 >> 172.21.15.97:0/105173984 conn(0x7f9b9c0fabb0 msgr2=0x7f9b9c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 shutdown_connections 2023-10-08T09:56:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:23.707+0000 7f9ba0d4c700 1 -- 172.21.15.97:0/105173984 wait complete. 2023-10-08T09:56:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:25 smithi097 ceph-mon[99491]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:25.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:25 smithi178 ceph-mon[103156]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:25.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:25 smithi186 ceph-mon[104393]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:27.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:27 smithi097 ceph-mon[99491]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-08T09:56:27.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:27 smithi178 ceph-mon[103156]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-08T09:56:27.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:27 smithi186 ceph-mon[104393]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-08T09:56:27.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1315251570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441007c0 msgr2=0x7f6a44100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1315251570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441007c0 0x7f6a44100be0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6a340099c0 tx=0x7f6a34009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1315251570 shutdown_connections 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1315251570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a44102350 0x7f6a44108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1315251570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441007c0 0x7f6a44100be0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1315251570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a441019b0 0x7f6a44101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1315251570 >> 172.21.15.97:0/1315251570 conn(0x7f6a440fbd30 msgr2=0x7f6a440fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1315251570 shutdown_connections 2023-10-08T09:56:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.889+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1315251570 wait complete. 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 Processor -- start 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 -- start start 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a441007c0 0x7f6a441967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 0x7f6a4419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419e310 con 0x7f6a441019b0 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419e450 con 0x7f6a441007c0 2023-10-08T09:56:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a49e6c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419e5e0 con 0x7f6a44102350 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57334/0 (socket says 172.21.15.97:57334) 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a42ffd700 1 -- 172.21.15.97:0/1110088846 learned_addr learned my addr 172.21.15.97:0/1110088846 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a437fe700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a441007c0 0x7f6a441967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.890+0000 7f6a43fff700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 0x7f6a4419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 -- 172.21.15.97:0/1110088846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 msgr2=0x7f6a4419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 0x7f6a4419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 -- 172.21.15.97:0/1110088846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a441007c0 msgr2=0x7f6a441967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a441007c0 0x7f6a441967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 -- 172.21.15.97:0/1110088846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340095f0 con 0x7f6a441019b0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a43fff700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 0x7f6a4419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a40ff9700 1 -- 172.21.15.97:0/1110088846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38009940 con 0x7f6a441019b0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a40ff9700 1 -- 172.21.15.97:0/1110088846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a38010460 con 0x7f6a441019b0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a40ff9700 1 -- 172.21.15.97:0/1110088846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800f620 con 0x7f6a441019b0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4419e8c0 con 0x7f6a441019b0 2023-10-08T09:56:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.891+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4419ed80 con 0x7f6a441019b0 2023-10-08T09:56:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.893+0000 7f6a40ff9700 1 -- 172.21.15.97:0/1110088846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6a380108d0 con 0x7f6a441019b0 2023-10-08T09:56:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.894+0000 7f6a40ff9700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 0x7f6a2c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.894+0000 7f6a437fe700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 0x7f6a2c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.894+0000 7f6a40ff9700 1 -- 172.21.15.97:0/1110088846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6a380ccea0 con 0x7f6a441019b0 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.894+0000 7f6a437fe700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 0x7f6a2c0a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009f50 tx=0x7f6a34005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 Processor -- start 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 -- start start 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 0x55dc16131480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 0x55dc16132c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16099460 con 0x55dc161319c0 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1608ac20 con 0x55dc16132800 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a56055b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16088540 con 0x55dc16131060 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57342/0 (socket says 172.21.15.97:57342) 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a42ffd700 1 -- 172.21.15.97:0/784529961 learned_addr learned my addr 172.21.15.97:0/784529961 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.895+0000 7f6a43fff700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 0x55dc16132c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a437fe700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 0x55dc16131480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 -- 172.21.15.97:0/784529961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 msgr2=0x55dc16131480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 0x55dc16131480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 -- 172.21.15.97:0/784529961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 msgr2=0x55dc16132c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 0x55dc16132c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 -- 172.21.15.97:0/784529961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc15eaede0 con 0x55dc161319c0 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a437fe700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 0x55dc16131480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a43fff700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 0x55dc16132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a1cf0 tx=0x7f6a38005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6479ace7cea820b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a29ffb700 1 -- 172.21.15.97:0/784529961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a2220 con 0x55dc161319c0 2023-10-08T09:56:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a29ffb700 1 -- 172.21.15.97:0/784529961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a38098230 con 0x55dc161319c0 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.896+0000 7f6a29ffb700 1 -- 172.21.15.97:0/784529961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a1d90 con 0x55dc161319c0 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- 172.21.15.97:0/784529961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 msgr2=0x55dc16132270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a380a1cf0 tx=0x7f6a38005910 comp rx=0 tx=0).stop 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- 172.21.15.97:0/784529961 shutdown_connections 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16132800 0x55dc16132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16131060 0x55dc16131480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 --2- 172.21.15.97:0/784529961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc16132270 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- 172.21.15.97:0/784529961 >> 172.21.15.97:0/784529961 conn(0x55dc160a5260 msgr2=0x55dc1612be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- 172.21.15.97:0/784529961 shutdown_connections 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- 172.21.15.97:0/784529961 wait complete. 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 -- start start 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.897+0000 7f6a56055b80 1 Processor -- start 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc161423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16083e30 con 0x55dc161319c0 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16020710 con 0x55dc16131060 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc160203b0 con 0x55dc16132800 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 10 client.0 _ll_register_callbacks cb 0x55dc15e7d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57348/0 (socket says 172.21.15.97:57348) 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 -- 172.21.15.97:0/3524886773 learned_addr learned my addr 172.21.15.97:0/3524886773 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a437fe700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a43fff700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc161423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 -- 172.21.15.97:0/3524886773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 msgr2=0x55dc161423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc161423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 -- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 msgr2=0x55dc1613ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 -- 172.21.15.97:0/3524886773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4419ed80 con 0x55dc161319c0 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a43fff700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc161423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a437fe700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:27.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a38099b90 tx=0x7f6a38009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855553dcb496678c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38099460 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6a380995c0 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc1604beb0 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a380a1d90 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.898+0000 7f6a56055b80 10 client.15777 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dc15e97c60 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc1604beb0 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6a38098df0 con 0x55dc161319c0 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:27.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 20 client.15777 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc16146680 0x55dc16148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 4 client.15777 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc1607ce10 con 0x55dc16146680 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.899+0000 7f6a437fe700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc16146680 0x55dc16148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a22ffd700 10 client.15777 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a437fe700 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc16146680 0x55dc16148ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4051923e559972 server_cookie=c86428f8adee5f5f in_seq=0 out_seq=0 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a22ffd700 1 -- 172.21.15.97:0/3524886773 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc1607ce10 con 0x55dc16146680 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a22ffd700 10 client.15777 handle_command_reply: tid=0 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a56055b80 1 client.15777 shutdown 2023-10-08T09:56:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 msgr2=0x55dc1613b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.900+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a38099b90 tx=0x7f6a38009650 comp rx=0 tx=0).stop 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 shutdown_connections 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc16146680 0x55dc16148ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc161423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a22ffd700 0 client.15777 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 --2- 172.21.15.97:0/3524886773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b370 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 >> 172.21.15.97:0/3524886773 conn(0x55dc160a5260 msgr2=0x55dc1612b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 shutdown_connections 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 1 -- 172.21.15.97:0/3524886773 wait complete. 2023-10-08T09:56:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.901+0000 7f6a56055b80 20 client.15777 trim_cache size 0 max 16384 2023-10-08T09:56:27.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 Processor -- start 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 -- start start 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16142370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc15f04920 con 0x55dc161319c0 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16132100 con 0x55dc16131060 2023-10-08T09:56:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.919+0000 7f6a56055b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1613a9a0 con 0x55dc16132800 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57356/0 (socket says 172.21.15.97:57356) 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 -- 172.21.15.97:0/4118471410 learned_addr learned my addr 172.21.15.97:0/4118471410 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a43fff700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a437fe700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 -- 172.21.15.97:0/4118471410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 msgr2=0x55dc16142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 -- 172.21.15.97:0/4118471410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 msgr2=0x55dc1613ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 -- 172.21.15.97:0/4118471410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc15ed5250 con 0x55dc161319c0 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a437fe700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009520 tx=0x7f6a380096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2888153dda354926 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/4118471410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a2220 con 0x55dc161319c0 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/4118471410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a38098df0 con 0x55dc161319c0 2023-10-08T09:56:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.920+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/4118471410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380995e0 con 0x55dc161319c0 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- 172.21.15.97:0/4118471410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 msgr2=0x55dc1613b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009520 tx=0x7f6a380096c0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- 172.21.15.97:0/4118471410 shutdown_connections 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc16131060 0x55dc1613ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 --2- 172.21.15.97:0/4118471410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc161319c0 0x55dc1613b350 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- 172.21.15.97:0/4118471410 >> 172.21.15.97:0/4118471410 conn(0x55dc160a5260 msgr2=0x55dc161357a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- 172.21.15.97:0/4118471410 shutdown_connections 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- 172.21.15.97:0/4118471410 wait complete. 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.921+0000 7f6a56055b80 1 -- start start 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 Processor -- start 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 0x55dc16141240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16141780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1619a4c0 con 0x55dc16131060 2023-10-08T09:56:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1619a600 con 0x55dc161319c0 2023-10-08T09:56:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1619a7c0 con 0x55dc16132800 2023-10-08T09:56:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a56055b80 10 client.0 _ll_register_callbacks cb 0x7f6a380997f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57360/0 (socket says 172.21.15.97:57360) 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 -- 172.21.15.97:0/939023358 learned_addr learned my addr 172.21.15.97:0/939023358 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a43fff700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 0x55dc16141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 -- 172.21.15.97:0/939023358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 msgr2=0x55dc16141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16141780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 -- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 msgr2=0x55dc16141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 0x55dc16141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a437fe700 1 -- 172.21.15.97:0/939023358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc15e97c60 con 0x55dc16131060 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a43fff700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.922+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 0x55dc16141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a437fe700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005850 tx=0x7f6a34009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120c42f57dcf4fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34033ad0 con 0x55dc16131060 2023-10-08T09:56:27.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6a34033c30 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a38098230 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a34033df0 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 10 client.15783 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dc15ead2b0 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a38098230 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6a34003c60 con 0x55dc16131060 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:27.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 20 client.15783 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc1619c2a0 0x55dc1619e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 4 client.15783 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.923+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc1607cb30 con 0x55dc1619c2a0 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc1619c2a0 0x55dc1619e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a22ffd700 10 client.15783 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a42ffd700 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc1619c2a0 0x55dc1619e6c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7edb92cf07ceb93 server_cookie=5d123d6abcb4e7ff in_seq=0 out_seq=0 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a22ffd700 1 -- 172.21.15.97:0/939023358 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dc1607cb30 con 0x55dc1619c2a0 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a22ffd700 10 client.15783 handle_command_reply: tid=0 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.924+0000 7f6a56055b80 1 client.15783 shutdown 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 msgr2=0x55dc16140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005850 tx=0x7f6a34009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 shutdown_connections 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55dc1619c2a0 0x55dc1619e6c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc16132800 0x55dc16141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a22ffd700 0 client.15783 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc16131060 0x55dc16140d00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 --2- 172.21.15.97:0/939023358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc161319c0 0x55dc16141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 >> 172.21.15.97:0/939023358 conn(0x55dc160a5260 msgr2=0x55dc1613add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 shutdown_connections 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 1 -- 172.21.15.97:0/939023358 wait complete. 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a56055b80 20 client.15783 trim_cache size 0 max 16384 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:56:27.938 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2023-10-08T09:56:27.939 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2023-10-08T09:56:27.939 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:56:27.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 msgr2=0x7f6a2c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 0x7f6a2c0a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009f50 tx=0x7f6a34005bc0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 msgr2=0x7f6a44196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.925+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dc60 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 shutdown_connections 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f6a2c0a1480 0x7f6a2c0a3940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a44102350 0x7f6a4419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a441007c0 0x7f6a441967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 --2- 172.21.15.97:0/1110088846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a441019b0 0x7f6a44196d20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:27.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 >> 172.21.15.97:0/1110088846 conn(0x7f6a440fbd30 msgr2=0x7f6a44104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 shutdown_connections 2023-10-08T09:56:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:27.926+0000 7f6a49e6c700 1 -- 172.21.15.97:0/1110088846 wait complete. 2023-10-08T09:56:27.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T09:56:27.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:56:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 -- 172.21.15.97:0/1748219975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 msgr2=0x7f1f000ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1748219975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f000ff9d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00099c0 tx=0x7f1ef0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 -- 172.21.15.97:0/1748219975 shutdown_connections 2023-10-08T09:56:28.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1748219975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f00101140 0x7f1f001079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1748219975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f000ff9d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1748219975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f001007a0 0x7f1f00100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 -- 172.21.15.97:0/1748219975 >> 172.21.15.97:0/1748219975 conn(0x7f1f000fab40 msgr2=0x7f1f000fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 -- 172.21.15.97:0/1748219975 shutdown_connections 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.070+0000 7f1f060e4700 1 -- 172.21.15.97:0/1748219975 wait complete. 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 Processor -- start 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 -- start start 2023-10-08T09:56:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f00069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101140 0x7f1f0019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019e100 con 0x7f1f000ff5b0 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019e270 con 0x7f1f001007a0 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1f060e4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0019e430 con 0x7f1f00101140 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1efeffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37346/0 (socket says 172.21.15.97:37346) 2023-10-08T09:56:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1efeffd700 1 -- 172.21.15.97:0/1931003177 learned_addr learned my addr 172.21.15.97:0/1931003177 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.071+0000 7f1efffff700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101140 0x7f1f0019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1eff7fe700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f00069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 -- 172.21.15.97:0/1931003177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101140 msgr2=0x7f1f0019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101140 0x7f1f0019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 -- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 msgr2=0x7f1f00069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f00069790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 -- 172.21.15.97:0/1931003177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00095f0 con 0x7f1f001007a0 2023-10-08T09:56:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efeffd700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400d950 tx=0x7f1ef400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4009940 con 0x7f1f001007a0 2023-10-08T09:56:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0019e710 con 0x7f1f001007a0 2023-10-08T09:56:28.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef4010460 con 0x7f1f001007a0 2023-10-08T09:56:28.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400f620 con 0x7f1f001007a0 2023-10-08T09:56:28.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.072+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0019ec70 con 0x7f1f001007a0 2023-10-08T09:56:28.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.073+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f001069d0 con 0x7f1f001007a0 2023-10-08T09:56:28.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.075+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1ef40108d0 con 0x7f1f001007a0 2023-10-08T09:56:28.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.075+0000 7f1efcff9700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 0x7f1ee80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.075+0000 7f1eff7fe700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 0x7f1ee80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.075+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1ef40ccec0 con 0x7f1f001007a0 2023-10-08T09:56:28.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.076+0000 7f1eff7fe700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 0x7f1ee80a3940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b580 tx=0x7f1ef0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.079+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ef40961f0 con 0x7f1f001007a0 2023-10-08T09:56:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.367+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f000678a0 con 0x7f1f001007a0 2023-10-08T09:56:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.368+0000 7f1efcff9700 1 -- 172.21.15.97:0/1931003177 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f1ef4020090 con 0x7f1f001007a0 2023-10-08T09:56:28.369 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:56:28.370 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:56:28.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.370+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 msgr2=0x7f1ee80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.370+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 0x7f1ee80a3940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000b580 tx=0x7f1ef0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.370+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 msgr2=0x7f1f0019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.370+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400d950 tx=0x7f1ef400dc60 comp rx=0 tx=0).stop 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 shutdown_connections 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1ee80a1480 0x7f1ee80a3940 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f00101140 0x7f1f0019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f000ff5b0 0x7f1f00069790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 --2- 172.21.15.97:0/1931003177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f001007a0 0x7f1f0019b310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 >> 172.21.15.97:0/1931003177 conn(0x7f1f000fab40 msgr2=0x7f1f00103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 shutdown_connections 2023-10-08T09:56:28.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.371+0000 7f1f060e4700 1 -- 172.21.15.97:0/1931003177 wait complete. 2023-10-08T09:56:28.375 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:56:28.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 -- 172.21.15.97:0/2703651210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 msgr2=0x7fadf4101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/2703651210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4101e10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faddc0099c0 tx=0x7faddc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 -- 172.21.15.97:0/2703651210 shutdown_connections 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/2703651210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf4102350 0x7fadf4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/2703651210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf41007c0 0x7fadf4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/2703651210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4101e10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 -- 172.21.15.97:0/2703651210 >> 172.21.15.97:0/2703651210 conn(0x7fadf40fbd30 msgr2=0x7fadf40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 -- 172.21.15.97:0/2703651210 shutdown_connections 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 -- 172.21.15.97:0/2703651210 wait complete. 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.513+0000 7fadfa2ec700 1 Processor -- start 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 -- start start 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf41007c0 0x7fadf4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 0x7fadf4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40738c0 con 0x7fadf41019b0 2023-10-08T09:56:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4073a30 con 0x7fadf4102350 2023-10-08T09:56:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadfa2ec700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf41a07c0 con 0x7fadf41007c0 2023-10-08T09:56:28.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57390/0 (socket says 172.21.15.97:57390) 2023-10-08T09:56:28.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 -- 172.21.15.97:0/121476566 learned_addr learned my addr 172.21.15.97:0/121476566 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadf3fff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf41007c0 0x7fadf4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadf8889700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 0x7fadf4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 -- 172.21.15.97:0/121476566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf41007c0 msgr2=0x7fadf4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf41007c0 0x7fadf4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 -- 172.21.15.97:0/121476566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 msgr2=0x7fadf4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 0x7fadf4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 -- 172.21.15.97:0/121476566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0095f0 con 0x7fadf41019b0 2023-10-08T09:56:28.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadf8889700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 0x7fadf4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:28.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadebfff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.514+0000 7fadf1ffb700 1 -- 172.21.15.97:0/121476566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc01d070 con 0x7fadf41019b0 2023-10-08T09:56:28.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.515+0000 7fadf1ffb700 1 -- 172.21.15.97:0/121476566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc003f90 con 0x7fadf41019b0 2023-10-08T09:56:28.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.515+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf41a09f0 con 0x7fadf41019b0 2023-10-08T09:56:28.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.515+0000 7fadf1ffb700 1 -- 172.21.15.97:0/121476566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc022d80 con 0x7fadf41019b0 2023-10-08T09:56:28.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.515+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf41a0db0 con 0x7fadf41019b0 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.517+0000 7fadf1ffb700 1 -- 172.21.15.97:0/121476566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faddc022470 con 0x7fadf41019b0 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.517+0000 7fadf1ffb700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 0x7fade00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.518+0000 7fadf3fff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 0x7fade00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.518+0000 7fadf1ffb700 1 -- 172.21.15.97:0/121476566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faddc0ce350 con 0x7fadf41019b0 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.518+0000 7fadf3fff700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 0x7fade00a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadf4101810 tx=0x7fade400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.518+0000 7fae064d5b80 1 Processor -- start 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.518+0000 7fae064d5b80 1 -- start start 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421ba9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba99c0 0x557421baa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421b11460 con 0x557421baa800 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421b02c20 con 0x557421ba9060 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fae064d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421b00540 con 0x557421ba99c0 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57400/0 (socket says 172.21.15.97:57400) 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 -- 172.21.15.97:0/1222922525 learned_addr learned my addr 172.21.15.97:0/1222922525 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadebfff700 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba99c0 0x557421baa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf3fff700 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421ba9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 -- 172.21.15.97:0/1222922525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba99c0 msgr2=0x557421baa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba99c0 0x557421baa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 -- 172.21.15.97:0/1222922525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 msgr2=0x557421ba9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421ba9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 -- 172.21.15.97:0/1222922525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557421926de0 con 0x557421baa800 2023-10-08T09:56:28.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadf8889700 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadec009e60 tx=0x7fadec0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6544c52d76f5135 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.519+0000 7fadea7fc700 1 -- 172.21.15.97:0/1222922525 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec014610 con 0x557421baa800 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fadea7fc700 1 -- 172.21.15.97:0/1222922525 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadec00f050 con 0x557421baa800 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fadea7fc700 1 -- 172.21.15.97:0/1222922525 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec013970 con 0x557421baa800 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 -- 172.21.15.97:0/1222922525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 msgr2=0x557421baac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadec009e60 tx=0x7fadec0077e0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 -- 172.21.15.97:0/1222922525 shutdown_connections 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421baac90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421ba9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1222922525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba99c0 0x557421baa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 -- 172.21.15.97:0/1222922525 >> 172.21.15.97:0/1222922525 conn(0x557421b1d260 msgr2=0x557421ba3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 -- 172.21.15.97:0/1222922525 shutdown_connections 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.520+0000 7fae064d5b80 1 -- 172.21.15.97:0/1222922525 wait complete. 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 -- start start 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 Processor -- start 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421bba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421afbe30 con 0x557421ba9060 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421a98710 con 0x557421ba99c0 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421a983b0 con 0x557421baa800 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fae064d5b80 10 client.0 _ll_register_callbacks cb 0x5574218f5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fadf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fadf3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57402/0 (socket says 172.21.15.97:57402) 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fadf3fff700 1 -- 172.21.15.97:0/4270392877 learned_addr learned my addr 172.21.15.97:0/4270392877 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fadebfff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.521+0000 7fadf8889700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421bba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 -- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 msgr2=0x557421bba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421bba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 -- 172.21.15.97:0/4270392877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 msgr2=0x557421bb3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 -- 172.21.15.97:0/4270392877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec009a90 con 0x557421ba9060 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadebfff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadf3fff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fade401d9e0 tx=0x7fade40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a40d230c24c31bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade401b850 con 0x557421ba9060 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fade401b9b0 con 0x557421ba9060 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557421ac3eb0 con 0x557421ba9060 2023-10-08T09:56:28.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade401bbb0 con 0x557421ba9060 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 10 client.15801 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55742190fc60 con 0x557421ba9060 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557421ac3eb0 con 0x557421ba9060 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fade40253f0 con 0x557421ba9060 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 20 client.15801 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421bbecb0 0x557421bc10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 4 client.15801 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.522+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557421af4e10 con 0x557421bbecb0 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.523+0000 7fadebfff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421bbecb0 0x557421bc10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.523+0000 7fadd37fe700 10 client.15801 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.523+0000 7fadebfff700 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421bbecb0 0x557421bc10d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ba276b21fb61249 server_cookie=7ab33f16b11ac7c1 in_seq=0 out_seq=0 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fadd37fe700 1 -- 172.21.15.97:0/4270392877 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557421af4e10 con 0x557421bbecb0 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fadd37fe700 10 client.15801 handle_command_reply: tid=0 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 client.15801 shutdown 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 msgr2=0x557421bb2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fade401d9e0 tx=0x7fade40047c0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 shutdown_connections 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421bbecb0 0x557421bc10d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421bba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba9060 0x557421bb2dd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fadd37fe700 0 client.15801 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:56:28.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 --2- 172.21.15.97:0/4270392877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 >> 172.21.15.97:0/4270392877 conn(0x557421b1d260 msgr2=0x557421ba34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 shutdown_connections 2023-10-08T09:56:28.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 1 -- 172.21.15.97:0/4270392877 wait complete. 2023-10-08T09:56:28.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.524+0000 7fae064d5b80 20 client.15801 trim_cache size 0 max 16384 2023-10-08T09:56:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.537+0000 7fae064d5b80 1 Processor -- start 2023-10-08T09:56:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 -- start start 2023-10-08T09:56:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421b9ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421ba3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742197c920 con 0x557421ba99c0 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421ba4310 con 0x557421ba9060 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fae064d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421bb0a60 con 0x557421baa800 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57412/0 (socket says 172.21.15.97:57412) 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 -- 172.21.15.97:0/3656173837 learned_addr learned my addr 172.21.15.97:0/3656173837 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadf8889700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421ba3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadf3fff700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421b9ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 -- 172.21.15.97:0/3656173837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 msgr2=0x557421ba3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421ba3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 -- 172.21.15.97:0/3656173837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 msgr2=0x557421b9ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421b9ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 -- 172.21.15.97:0/3656173837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55742194d250 con 0x557421ba99c0 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadf3fff700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421b9ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadebfff700 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faddc0978d0 tx=0x7faddc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425b99c70e2c9330 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadeaffd700 1 -- 172.21.15.97:0/3656173837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0b7070 con 0x557421ba99c0 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadeaffd700 1 -- 172.21.15.97:0/3656173837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc09f520 con 0x557421ba99c0 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.538+0000 7fadeaffd700 1 -- 172.21.15.97:0/3656173837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc004100 con 0x557421ba99c0 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 -- 172.21.15.97:0/3656173837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 msgr2=0x557421bba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faddc0978d0 tx=0x7faddc01a040 comp rx=0 tx=0).stop 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 -- 172.21.15.97:0/3656173837 shutdown_connections 2023-10-08T09:56:28.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421baa800 0x557421ba3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba9060 0x557421b9ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 --2- 172.21.15.97:0/3656173837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421ba99c0 0x557421bba310 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 -- 172.21.15.97:0/3656173837 >> 172.21.15.97:0/3656173837 conn(0x557421b1d260 msgr2=0x557421bc5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 -- 172.21.15.97:0/3656173837 shutdown_connections 2023-10-08T09:56:28.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.539+0000 7fae064d5b80 1 -- 172.21.15.97:0/3656173837 wait complete. 2023-10-08T09:56:28.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 -- start start 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 Processor -- start 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba9060 0x557421bb8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421bb9d50 con 0x557421baa800 2023-10-08T09:56:28.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421bb9ec0 con 0x557421ba99c0 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557421bba080 con 0x557421ba9060 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fae064d5b80 10 client.0 _ll_register_callbacks cb 0x7fade400ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57420/0 (socket says 172.21.15.97:57420) 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 -- 172.21.15.97:0/1660769503 learned_addr learned my addr 172.21.15.97:0/1660769503 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf3fff700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba9060 0x557421bb8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadebfff700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 -- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba9060 msgr2=0x557421bb8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba9060 0x557421bb8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 -- 172.21.15.97:0/1660769503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 msgr2=0x557421bb88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.540+0000 7fadf8889700 1 -- 172.21.15.97:0/1660769503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf41a0db0 con 0x557421baa800 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadf8889700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadec000f80 tx=0x7fadec005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d3ee3c102f5a82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec022070 con 0x557421baa800 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadec00f050 con 0x557421baa800 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fadec005570 con 0x557421baa800 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadec006000 con 0x557421baa800 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 10 client.15807 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5574219252b0 con 0x557421baa800 2023-10-08T09:56:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadec00f050 con 0x557421baa800 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadec014ca0 con 0x557421baa800 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 10 client.15807 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 10 client.15807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 20 client.15807 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421c13c10 0x557421c16030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 4 client.15807 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.541+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557421af4b30 con 0x557421c13c10 2023-10-08T09:56:28.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fadf3fff700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421c13c10 0x557421c16030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fadd37fe700 10 client.15807 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fadf3fff700 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421c13c10 0x557421c16030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9664bf61074362f server_cookie=38469fe4e83d2438 in_seq=0 out_seq=0 2023-10-08T09:56:28.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fadd37fe700 1 -- 172.21.15.97:0/1660769503 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557421af4b30 con 0x557421c13c10 2023-10-08T09:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fadd37fe700 10 client.15807 handle_command_reply: tid=0 2023-10-08T09:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fae064d5b80 1 client.15807 shutdown 2023-10-08T09:56:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 msgr2=0x557421bb9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.542+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fadec000f80 tx=0x7fadec005270 comp rx=0 tx=0).stop 2023-10-08T09:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 shutdown_connections 2023-10-08T09:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x557421c13c10 0x557421c16030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557421baa800 0x557421bb9810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557421ba9060 0x557421bb8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 --2- 172.21.15.97:0/1660769503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557421ba99c0 0x557421bb88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 >> 172.21.15.97:0/1660769503 conn(0x557421b1d260 msgr2=0x557421ba2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadd37fe700 0 client.15807 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:56:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 shutdown_connections 2023-10-08T09:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 1 -- 172.21.15.97:0/1660769503 wait complete. 2023-10-08T09:56:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fae064d5b80 20 client.15807 trim_cache size 0 max 16384 2023-10-08T09:56:28.559 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 msgr2=0x7fade00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 0x7fade00a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadf4101810 tx=0x7fade400a380 comp rx=0 tx=0).stop 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 msgr2=0x7fadf4072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc004730 comp rx=0 tx=0).stop 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 shutdown_connections 2023-10-08T09:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fade00a1480 0x7fade00a3940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf4102350 0x7fadf4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadf41007c0 0x7fadf4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 --2- 172.21.15.97:0/121476566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf41019b0 0x7fadf4072be0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 >> 172.21.15.97:0/121476566 conn(0x7fadf40fbd30 msgr2=0x7fadf4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 shutdown_connections 2023-10-08T09:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.543+0000 7fadfa2ec700 1 -- 172.21.15.97:0/121476566 wait complete. 2023-10-08T09:56:28.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T09:56:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.689+0000 7f67622c3700 1 -- 172.21.15.97:0/2717914138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c102350 msgr2=0x7f675c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.689+0000 7f67622c3700 1 --2- 172.21.15.97:0/2717914138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c102350 0x7f675c108c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f675400b0d0 tx=0x7f675400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 -- 172.21.15.97:0/2717914138 shutdown_connections 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- 172.21.15.97:0/2717914138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c102350 0x7f675c108c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- 172.21.15.97:0/2717914138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c1007c0 0x7f675c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- 172.21.15.97:0/2717914138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1019b0 0x7f675c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 -- 172.21.15.97:0/2717914138 >> 172.21.15.97:0/2717914138 conn(0x7f675c0fbd30 msgr2=0x7f675c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 -- 172.21.15.97:0/2717914138 shutdown_connections 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 -- 172.21.15.97:0/2717914138 wait complete. 2023-10-08T09:56:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 Processor -- start 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 -- start start 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1007c0 0x7f675c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.690+0000 7f67622c3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 0x7f675c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67622c3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c19e340 con 0x7f675c1019b0 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67622c3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c19e480 con 0x7f675c1007c0 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67622c3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c19e610 con 0x7f675c102350 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57432/0 (socket says 172.21.15.97:57432) 2023-10-08T09:56:28.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 -- 172.21.15.97:0/3597271414 learned_addr learned my addr 172.21.15.97:0/3597271414 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675bfff700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1007c0 0x7f675c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f6760860700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 0x7f675c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 -- 172.21.15.97:0/3597271414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 msgr2=0x7f675c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 0x7f675c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 -- 172.21.15.97:0/3597271414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1007c0 msgr2=0x7f675c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1007c0 0x7f675c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 -- 172.21.15.97:0/3597271414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009d00 con 0x7f675c1019b0 2023-10-08T09:56:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f6760860700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 0x7f675c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f675b7fe700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f675000d950 tx=0x7f675000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67597fa700 1 -- 172.21.15.97:0/3597271414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750009940 con 0x7f675c1019b0 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c19e8f0 con 0x7f675c1019b0 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67597fa700 1 -- 172.21.15.97:0/3597271414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6750010460 con 0x7f675c1019b0 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c19ee50 con 0x7f675c1019b0 2023-10-08T09:56:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.691+0000 7f67597fa700 1 -- 172.21.15.97:0/3597271414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000f620 con 0x7f675c1019b0 2023-10-08T09:56:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f67597fa700 1 -- 172.21.15.97:0/3597271414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f67500108d0 con 0x7f675c1019b0 2023-10-08T09:56:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 Processor -- start 2023-10-08T09:56:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 -- start start 2023-10-08T09:56:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2007270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2007c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f6e460 con 0x5596e2006060 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f5fc20 con 0x5596e2007800 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f676e4acb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f5d540 con 0x5596e20069c0 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f67597fa700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 0x7f67440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f6760860700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f6760860700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57436/0 (socket says 172.21.15.97:57436) 2023-10-08T09:56:28.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f6760860700 1 -- 172.21.15.97:0/4146723064 learned_addr learned my addr 172.21.15.97:0/4146723064 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f6760860700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 0x7f67440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f675b7fe700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2007c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f67597fa700 1 -- 172.21.15.97:0/3597271414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f67500cd240 con 0x7f675c1019b0 2023-10-08T09:56:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.695+0000 7f675bfff700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2007270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 -- 172.21.15.97:0/4146723064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 msgr2=0x5596e2007270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2007270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 -- 172.21.15.97:0/4146723064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 msgr2=0x5596e2007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2007c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 -- 172.21.15.97:0/4146723064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e1d83de0 con 0x5596e2006060 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f675b7fe700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2007c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f675bfff700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2007270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f675400b0d0 tx=0x7f6754009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109e8b396f12fdd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f6760860700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 0x7f67440a3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6754015000 tx=0x7f6754019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f67427fc700 1 -- 172.21.15.97:0/4146723064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400e040 con 0x5596e2006060 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f67427fc700 1 -- 172.21.15.97:0/4146723064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6754008d30 con 0x5596e2006060 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.696+0000 7f67427fc700 1 -- 172.21.15.97:0/4146723064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754025070 con 0x5596e2006060 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- 172.21.15.97:0/4146723064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 msgr2=0x5596e2006480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f675400b0d0 tx=0x7f6754009000 comp rx=0 tx=0).stop 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- 172.21.15.97:0/4146723064 shutdown_connections 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2007c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006480 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 --2- 172.21.15.97:0/4146723064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2007270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- 172.21.15.97:0/4146723064 >> 172.21.15.97:0/4146723064 conn(0x5596e1f7a260 msgr2=0x5596e2000e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- 172.21.15.97:0/4146723064 shutdown_connections 2023-10-08T09:56:28.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- 172.21.15.97:0/4146723064 wait complete. 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.697+0000 7f676e4acb80 1 -- start start 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 Processor -- start 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e2006060 0x5596e200fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e20173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f58e30 con 0x5596e20069c0 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1ef5710 con 0x5596e2007800 2023-10-08T09:56:28.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1ef53b0 con 0x5596e2006060 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f676e4acb80 10 client.0 _ll_register_callbacks cb 0x5596e1d52800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57446/0 (socket says 172.21.15.97:57446) 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 -- 172.21.15.97:0/1059172356 learned_addr learned my addr 172.21.15.97:0/1059172356 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675bfff700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e2006060 0x5596e200fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f6760860700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e20173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 -- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e2006060 msgr2=0x5596e200fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e2006060 0x5596e200fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 -- 172.21.15.97:0/1059172356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 msgr2=0x5596e20173d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e20173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 -- 172.21.15.97:0/1059172356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c19ee50 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f675b7fe700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6750096ca0 tx=0x7f6750018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a972befab2ef0b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.698+0000 7f673b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675009a930 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6750094e40 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596e1f20eb0 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f675009b320 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 10 client.15819 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f675c109550 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596e1f20eb0 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6750021d30 con 0x5596e20069c0 2023-10-08T09:56:28.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 10 client.15819 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 20 client.15819 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e201bc70 0x5596e201e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 4 client.15819 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596e1f51e10 con 0x5596e201bc70 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.699+0000 7f675bfff700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e201bc70 0x5596e201e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.700+0000 7f673b7fe700 10 client.15819 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.700+0000 7f675bfff700 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e201bc70 0x5596e201e090 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=895a1e0a516e2552 server_cookie=155b932808c6591f in_seq=0 out_seq=0 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f673b7fe700 1 -- 172.21.15.97:0/1059172356 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596e1f51e10 con 0x5596e201bc70 2023-10-08T09:56:28.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f673b7fe700 10 client.15819 handle_command_reply: tid=0 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 client.15819 shutdown 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 msgr2=0x5596e20103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6750096ca0 tx=0x7f6750018f70 comp rx=0 tx=0).stop 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 shutdown_connections 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e201bc70 0x5596e201e090 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e20173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e2006060 0x5596e200fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f673b7fe700 0 client.15819 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 --2- 172.21.15.97:0/1059172356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e20069c0 0x5596e20103e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 >> 172.21.15.97:0/1059172356 conn(0x5596e1f7a260 msgr2=0x5596e20004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 shutdown_connections 2023-10-08T09:56:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 1 -- 172.21.15.97:0/1059172356 wait complete. 2023-10-08T09:56:28.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.701+0000 7f676e4acb80 20 client.15819 trim_cache size 0 max 16384 2023-10-08T09:56:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 Processor -- start 2023-10-08T09:56:28.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 -- start start 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2017390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e1ffcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1dd9920 con 0x5596e2006060 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f51e10 con 0x5596e2007800 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f676e4acb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e1f20eb0 con 0x5596e20069c0 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57452/0 (socket says 172.21.15.97:57452) 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 -- 172.21.15.97:0/3730131690 learned_addr learned my addr 172.21.15.97:0/3730131690 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f6760860700 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e1ffcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 -- 172.21.15.97:0/3730131690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 msgr2=0x5596e2017390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2017390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 -- 172.21.15.97:0/3730131690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 msgr2=0x5596e1ffcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e1ffcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 -- 172.21.15.97:0/3730131690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e1d6cc60 con 0x5596e2006060 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f6760860700 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e1ffcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.715+0000 7f675bfff700 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5596e201e970 tx=0x7f674c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18630b1bbb50e4c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f6742ffd700 1 -- 172.21.15.97:0/3730131690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c01b5b0 con 0x5596e2006060 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f6742ffd700 1 -- 172.21.15.97:0/3730131690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c016040 con 0x5596e2006060 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f6742ffd700 1 -- 172.21.15.97:0/3730131690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c005280 con 0x5596e2006060 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 -- 172.21.15.97:0/3730131690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 msgr2=0x5596e2006440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x5596e201e970 tx=0x7f674c009260 comp rx=0 tx=0).stop 2023-10-08T09:56:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 -- 172.21.15.97:0/3730131690 shutdown_connections 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e1ffcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2006440 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 --2- 172.21.15.97:0/3730131690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e2017390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 -- 172.21.15.97:0/3730131690 >> 172.21.15.97:0/3730131690 conn(0x5596e1f7a260 msgr2=0x5596e2000dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 -- 172.21.15.97:0/3730131690 shutdown_connections 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.716+0000 7f676e4acb80 1 -- 172.21.15.97:0/3730131690 wait complete. 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 -- start start 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 Processor -- start 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e20172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2015560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2015c70 con 0x5596e2006060 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2015de0 con 0x5596e2007800 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2015fa0 con 0x5596e20069c0 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f675bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f676e4acb80 10 client.0 _ll_register_callbacks cb 0x7f67500965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.717+0000 7f675bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57458/0 (socket says 172.21.15.97:57458) 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 -- 172.21.15.97:0/2545923127 learned_addr learned my addr 172.21.15.97:0/2545923127 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675b7fe700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e20172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f6760860700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2015560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 -- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 msgr2=0x5596e20172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e20172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 -- 172.21.15.97:0/2545923127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 msgr2=0x5596e2015560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2015560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 -- 172.21.15.97:0/2545923127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c009b70 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f675bfff700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f674c001eb0 tx=0x7f674c01b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b5b550fa52339a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f673b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c01b910 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f674c000e40 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6754008d30 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674c01ba70 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f676e4acb80 10 client.15825 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5596e1daa250 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.718+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6754008d30 con 0x5596e2006060 2023-10-08T09:56:28.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f674c003780 con 0x5596e2006060 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 10 client.15825 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 20 client.15825 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e2070e50 0x5596e2073270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 4 client.15825 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596e1f51b30 con 0x5596e2070e50 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f675b7fe700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e2070e50 0x5596e2073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f673b7fe700 10 client.15825 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f675b7fe700 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e2070e50 0x5596e2073270 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=607b7a5f9351e951 server_cookie=d395c76fce5d5fe6 in_seq=0 out_seq=0 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.719+0000 7f673b7fe700 1 -- 172.21.15.97:0/2545923127 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596e1f51b30 con 0x5596e2070e50 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f673b7fe700 10 client.15825 handle_command_reply: tid=0 2023-10-08T09:56:28.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 client.15825 shutdown 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 msgr2=0x5596e2016da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f674c001eb0 tx=0x7f674c01b510 comp rx=0 tx=0).stop 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 shutdown_connections 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x5596e2070e50 0x5596e2073270 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e2007800 0x5596e2015560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596e2006060 0x5596e2016da0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 --2- 172.21.15.97:0/2545923127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5596e20069c0 0x5596e20172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f673b7fe700 0 client.15825 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 >> 172.21.15.97:0/2545923127 conn(0x5596e1f7a260 msgr2=0x5596e2000d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 shutdown_connections 2023-10-08T09:56:28.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 1 -- 172.21.15.97:0/2545923127 wait complete. 2023-10-08T09:56:28.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f676e4acb80 20 client.15825 trim_cache size 0 max 16384 2023-10-08T09:56:28.734 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:56:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 msgr2=0x7f67440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 0x7f67440a3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6754015000 tx=0x7f6754019000 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 msgr2=0x7f675c196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.720+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f675000d950 tx=0x7f675000dc60 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 shutdown_connections 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f67440a1480 0x7f67440a3940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f675c102350 0x7f675c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f675c1007c0 0x7f675c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 --2- 172.21.15.97:0/3597271414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f675c1019b0 0x7f675c196d50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 >> 172.21.15.97:0/3597271414 conn(0x7f675c0fbd30 msgr2=0x7f675c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 shutdown_connections 2023-10-08T09:56:28.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.721+0000 7f67622c3700 1 -- 172.21.15.97:0/3597271414 wait complete. 2023-10-08T09:56:28.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T09:56:28.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 -- 172.21.15.97:0/2320593840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc100890 msgr2=0x7f24bc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 --2- 172.21.15.97:0/2320593840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc100890 0x7f24bc100cf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f24b00099c0 tx=0x7f24b0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 -- 172.21.15.97:0/2320593840 shutdown_connections 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 --2- 172.21.15.97:0/2320593840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc101230 0x7f24bc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 --2- 172.21.15.97:0/2320593840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc0ff6a0 0x7f24bc0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 --2- 172.21.15.97:0/2320593840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc100890 0x7f24bc100cf0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 -- 172.21.15.97:0/2320593840 >> 172.21.15.97:0/2320593840 conn(0x7f24bc0fac10 msgr2=0x7f24bc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 -- 172.21.15.97:0/2320593840 shutdown_connections 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.862+0000 7f24c22ed700 1 -- 172.21.15.97:0/2320593840 wait complete. 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 Processor -- start 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 -- start start 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc0ff6a0 0x7f24bc068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc100890 0x7f24bc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19e1d0 con 0x7f24bc101230 2023-10-08T09:56:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19e340 con 0x7f24bc100890 2023-10-08T09:56:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c22ed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc19e500 con 0x7f24bc0ff6a0 2023-10-08T09:56:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c088a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c088a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57476/0 (socket says 172.21.15.97:57476) 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24c088a700 1 -- 172.21.15.97:0/3937961271 learned_addr learned my addr 172.21.15.97:0/3937961271 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc100890 0x7f24bc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.863+0000 7f24bbfff700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc0ff6a0 0x7f24bc068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 -- 172.21.15.97:0/3937961271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc0ff6a0 msgr2=0x7f24bc068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc0ff6a0 0x7f24bc068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 -- 172.21.15.97:0/3937961271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc100890 msgr2=0x7f24bc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc100890 0x7f24bc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 -- 172.21.15.97:0/3937961271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00095f0 con 0x7f24bc101230 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c088a700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f24b4007ee0 tx=0x7f24b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24b97fa700 1 -- 172.21.15.97:0/3937961271 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4017070 con 0x7f24bc101230 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc19e7e0 con 0x7f24bc101230 2023-10-08T09:56:28.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24b97fa700 1 -- 172.21.15.97:0/3937961271 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b400f040 con 0x7f24bc101230 2023-10-08T09:56:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24b97fa700 1 -- 172.21.15.97:0/3937961271 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4005580 con 0x7f24bc101230 2023-10-08T09:56:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.864+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc19ec70 con 0x7f24bc101230 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.867+0000 7f24b97fa700 1 -- 172.21.15.97:0/3937961271 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f24b400db20 con 0x7f24bc101230 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.867+0000 7f24b97fa700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 0x7f24a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.867+0000 7f24b97fa700 1 -- 172.21.15.97:0/3937961271 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f24b40cc700 con 0x7f24bc101230 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.867+0000 7f24bbfff700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 0x7f24a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24bbfff700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 0x7f24a40a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24bc1006f0 tx=0x7f24ac007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 Processor -- start 2023-10-08T09:56:28.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 -- start start 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec69e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec69fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec606460 con 0x55e8ec69e9c0 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec5f7c20 con 0x55e8ec69f800 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec5f5540 con 0x55e8ec69e060 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57482/0 (socket says 172.21.15.97:57482) 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24bb7fe700 1 -- 172.21.15.97:0/1081471490 learned_addr learned my addr 172.21.15.97:0/1081471490 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24bbfff700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec69e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.868+0000 7f24c088a700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec69fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 -- 172.21.15.97:0/1081471490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 msgr2=0x55e8ec69e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec69e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 -- 172.21.15.97:0/1081471490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 msgr2=0x55e8ec69fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec69fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 -- 172.21.15.97:0/1081471490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ec41bde0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24c088a700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec69fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b580 tx=0x7f24b0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ee58d61a5e8953 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24a27fc700 1 -- 172.21.15.97:0/1081471490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d070 con 0x55e8ec69e9c0 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24a27fc700 1 -- 172.21.15.97:0/1081471490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b001e040 con 0x55e8ec69e9c0 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.869+0000 7f24a27fc700 1 -- 172.21.15.97:0/1081471490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0003b20 con 0x55e8ec69e9c0 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1081471490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 msgr2=0x55e8ec69f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b580 tx=0x7f24b0005bc0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1081471490 shutdown_connections 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec69fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec69e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1081471490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec69f270 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1081471490 >> 172.21.15.97:0/1081471490 conn(0x55e8ec612260 msgr2=0x55e8ec698e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1081471490 shutdown_connections 2023-10-08T09:56:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1081471490 wait complete. 2023-10-08T09:56:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.870+0000 7f24ce4d6b80 1 -- start start 2023-10-08T09:56:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 Processor -- start 2023-10-08T09:56:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec6af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec5f0e30 con 0x55e8ec69e9c0 2023-10-08T09:56:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec58d710 con 0x55e8ec69f800 2023-10-08T09:56:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec58d3b0 con 0x55e8ec69e060 2023-10-08T09:56:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24ce4d6b80 10 client.0 _ll_register_callbacks cb 0x55e8ec3ea800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57488/0 (socket says 172.21.15.97:57488) 2023-10-08T09:56:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 -- 172.21.15.97:0/814705249 learned_addr learned my addr 172.21.15.97:0/814705249 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bbfff700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24c088a700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec6af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 -- 172.21.15.97:0/814705249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 msgr2=0x55e8ec6a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 -- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 msgr2=0x55e8ec6af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec6af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.871+0000 7f24bb7fe700 1 -- 172.21.15.97:0/814705249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc108430 con 0x55e8ec69e9c0 2023-10-08T09:56:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f24b0006010 tx=0x7f24b0005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8597ae073f52ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d030 con 0x55e8ec69e9c0 2023-10-08T09:56:28.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f24b0003ac0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8ec5b8eb0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b000bea0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 10 client.15837 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e8ec404c60 con 0x55e8ec69e9c0 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8ec5b8eb0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f24b0004bc0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 20 client.15837 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec6b3760 0x55e8ec6b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 4 client.15837 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.872+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8ec5e9e10 con 0x55e8ec6b3760 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.873+0000 7f24bbfff700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec6b3760 0x55e8ec6b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.873+0000 7f24bbfff700 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec6b3760 0x55e8ec6b5b80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a8e617849352df server_cookie=f49ef35f0c7fc384 in_seq=0 out_seq=0 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.873+0000 7f249b7fe700 10 client.15837 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:28.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f249b7fe700 1 -- 172.21.15.97:0/814705249 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8ec5e9e10 con 0x55e8ec6b3760 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f249b7fe700 10 client.15837 handle_command_reply: tid=0 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 client.15837 shutdown 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 msgr2=0x55e8ec6a82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f24b0006010 tx=0x7f24b0005470 comp rx=0 tx=0).stop 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 shutdown_connections 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69f800 0x55e8ec6af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec6b3760 0x55e8ec6b5b80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f249b7fe700 0 client.15837 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/814705249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6a82b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 >> 172.21.15.97:0/814705249 conn(0x55e8ec612260 msgr2=0x55e8ec6984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 shutdown_connections 2023-10-08T09:56:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/814705249 wait complete. 2023-10-08T09:56:28.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.874+0000 7f24ce4d6b80 20 client.15837 trim_cache size 0 max 16384 2023-10-08T09:56:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 Processor -- start 2023-10-08T09:56:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 -- start start 2023-10-08T09:56:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec471920 con 0x55e8ec69f800 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec6a78e0 con 0x55e8ec69e060 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec695ae0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24c088a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24c088a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57498/0 (socket says 172.21.15.97:57498) 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24c088a700 1 -- 172.21.15.97:0/1783592427 learned_addr learned my addr 172.21.15.97:0/1783592427 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.887+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24bbfff700 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 -- 172.21.15.97:0/1783592427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e9c0 msgr2=0x55e8ec6af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 -- 172.21.15.97:0/1783592427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 msgr2=0x55e8ec6a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 -- 172.21.15.97:0/1783592427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ec442250 con 0x55e8ec69f800 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24c088a700 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24b4095bf0 tx=0x7f24b4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be6ef00fd1060a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24a2ffd700 1 -- 172.21.15.97:0/1783592427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40b90c0 con 0x55e8ec69f800 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24a2ffd700 1 -- 172.21.15.97:0/1783592427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b40996f0 con 0x55e8ec69f800 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.888+0000 7f24a2ffd700 1 -- 172.21.15.97:0/1783592427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40840b0 con 0x55e8ec69f800 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1783592427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 msgr2=0x55e8ec694f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24b4095bf0 tx=0x7f24b4084040 comp rx=0 tx=0).stop 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1783592427 shutdown_connections 2023-10-08T09:56:28.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69f800 0x55e8ec694f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/1783592427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1783592427 >> 172.21.15.97:0/1783592427 conn(0x55e8ec612260 msgr2=0x55e8ec6ba4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1783592427 shutdown_connections 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/1783592427 wait complete. 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 -- start start 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.889+0000 7f24ce4d6b80 1 Processor -- start 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69f800 0x55e8ec6ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec6abc00 con 0x55e8ec69e9c0 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec6abd70 con 0x55e8ec69e060 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ec6abf30 con 0x55e8ec69f800 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 10 client.0 _ll_register_callbacks cb 0x7f24b0005920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57502/0 (socket says 172.21.15.97:57502) 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 -- 172.21.15.97:0/2434981141 learned_addr learned my addr 172.21.15.97:0/2434981141 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24c088a700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69f800 0x55e8ec6ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bbfff700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 -- 172.21.15.97:0/2434981141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69f800 msgr2=0x55e8ec6ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69f800 0x55e8ec6ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 -- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 msgr2=0x55e8ec6aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 -- 172.21.15.97:0/2434981141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc19ec70 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bbfff700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24bb7fe700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b00098e0 tx=0x7f24b0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2a7d0c4c495086 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f249b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d070 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b001e040 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f24b0003ad0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b00054e0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 10 client.15843 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.890+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e8ec41a2b0 con 0x55e8ec69e9c0 2023-10-08T09:56:28.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b001e040 con 0x55e8ec69e9c0 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f24b0004500 con 0x55e8ec69e9c0 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 20 client.15843 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec7085a0 0x55e8ec70a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 4 client.15843 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8ec5e9b30 con 0x55e8ec7085a0 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.891+0000 7f24bbfff700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec7085a0 0x55e8ec70a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24bbfff700 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec7085a0 0x55e8ec70a9c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41343f5e1588b4a7 server_cookie=cbf1924b11b40c5d in_seq=0 out_seq=0 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f249b7fe700 10 client.15843 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:28.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f249b7fe700 1 -- 172.21.15.97:0/2434981141 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8ec5e9b30 con 0x55e8ec7085a0 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f249b7fe700 10 client.15843 handle_command_reply: tid=0 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 client.15843 shutdown 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 msgr2=0x55e8ec6af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b00098e0 tx=0x7f24b0005bc0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 shutdown_connections 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8ec7085a0 0x55e8ec70a9c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ec69f800 0x55e8ec6ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8ec69e060 0x55e8ec6aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f249b7fe700 0 client.15843 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 --2- 172.21.15.97:0/2434981141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ec69e9c0 0x55e8ec6af0c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 >> 172.21.15.97:0/2434981141 conn(0x55e8ec612260 msgr2=0x55e8ec694f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 shutdown_connections 2023-10-08T09:56:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 1 -- 172.21.15.97:0/2434981141 wait complete. 2023-10-08T09:56:28.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.892+0000 7f24ce4d6b80 20 client.15843 trim_cache size 0 max 16384 2023-10-08T09:56:28.910 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:56:28.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 msgr2=0x7f24a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 0x7f24a40a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24bc1006f0 tx=0x7f24ac007400 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 msgr2=0x7f24bc19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f24b4007ee0 tx=0x7f24b400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 shutdown_connections 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f24a40a1480 0x7f24a40a3940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24bc101230 0x7f24bc19da30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f24bc0ff6a0 0x7f24bc068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 --2- 172.21.15.97:0/3937961271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24bc100890 0x7f24bc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 >> 172.21.15.97:0/3937961271 conn(0x7f24bc0fac10 msgr2=0x7f24bc103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 shutdown_connections 2023-10-08T09:56:28.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:28.893+0000 7f24c22ed700 1 -- 172.21.15.97:0/3937961271 wait complete. 2023-10-08T09:56:29.190 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:29 smithi097 ceph-mon[99491]: pgmap v340: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:29.191 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:29 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1931003177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:56:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:29 smithi178 ceph-mon[103156]: pgmap v340: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:29.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:29 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1931003177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:56:29.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:29 smithi186 ceph-mon[104393]: pgmap v340: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:29.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:29 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1931003177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:56:29.914 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:56:29.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:56:30.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 -- 172.21.15.97:0/627152389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 msgr2=0x7fe324108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 --2- 172.21.15.97:0/627152389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe324108900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00b0a0 tx=0x7fe31c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 -- 172.21.15.97:0/627152389 shutdown_connections 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 --2- 172.21.15.97:0/627152389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe324108900 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 --2- 172.21.15.97:0/627152389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 0x7fe324103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 --2- 172.21.15.97:0/627152389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe324103bc0 0x7fe324105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 -- 172.21.15.97:0/627152389 >> 172.21.15.97:0/627152389 conn(0x7fe3240fac40 msgr2=0x7fe3240fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 -- 172.21.15.97:0/627152389 shutdown_connections 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.083+0000 7fe329559700 1 -- 172.21.15.97:0/627152389 wait complete. 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 Processor -- start 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 -- start start 2023-10-08T09:56:30.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 0x7fe3241968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe324103bc0 0x7fe324196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e380 con 0x7fe3241064f0 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e4c0 con 0x7fe324103bc0 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.084+0000 7fe329559700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419e650 con 0x7fe324101290 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57516/0 (socket says 172.21.15.97:57516) 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 -- 172.21.15.97:0/858298952 learned_addr learned my addr 172.21.15.97:0/858298952 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe322ffd700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 0x7fe3241968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 -- 172.21.15.97:0/858298952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 msgr2=0x7fe3241968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 0x7fe3241968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 -- 172.21.15.97:0/858298952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe324103bc0 msgr2=0x7fe324196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe324103bc0 0x7fe324196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 -- 172.21.15.97:0/858298952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c009d00 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe3237fe700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007ab0 tx=0x7fe31c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe30bfff700 1 -- 172.21.15.97:0/858298952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00e040 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe30bfff700 1 -- 172.21.15.97:0/858298952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe31c005710 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe30bfff700 1 -- 172.21.15.97:0/858298952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c01dae0 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32419e8d0 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.085+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32419eca0 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.088+0000 7fe30bfff700 1 -- 172.21.15.97:0/858298952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe31c005880 con 0x7fe3241064f0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.089+0000 7fe30bfff700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 0x7fe30c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.089+0000 7fe322ffd700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 0x7fe30c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.089+0000 7fe30bfff700 1 -- 172.21.15.97:0/858298952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe31c0ce610 con 0x7fe3241064f0 2023-10-08T09:56:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.089+0000 7fe322ffd700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 0x7fe30c0a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe314005950 tx=0x7fe31400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 Processor -- start 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 -- start start 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec5372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec537d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec49e4e0 con 0x563cec5360e0 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec48fca0 con 0x563cec537880 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe335742b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec48d5c0 con 0x563cec536a40 2023-10-08T09:56:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57522/0 (socket says 172.21.15.97:57522) 2023-10-08T09:56:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 -- 172.21.15.97:0/3092169019 learned_addr learned my addr 172.21.15.97:0/3092169019 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe3237fe700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec537d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe3227fc700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec5372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 -- 172.21.15.97:0/3092169019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 msgr2=0x563cec5372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec5372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 -- 172.21.15.97:0/3092169019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 msgr2=0x563cec537d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec537d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.090+0000 7fe322ffd700 1 -- 172.21.15.97:0/3092169019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cec2b3e60 con 0x563cec5360e0 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.091+0000 7fe3227fc700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec5372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.091+0000 7fe322ffd700 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe314010fa0 tx=0x7fe314013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263db30ca075d1a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.091+0000 7fe3097fa700 1 -- 172.21.15.97:0/3092169019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314011db0 con 0x563cec5360e0 2023-10-08T09:56:30.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.091+0000 7fe3097fa700 1 -- 172.21.15.97:0/3092169019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe314015070 con 0x563cec5360e0 2023-10-08T09:56:30.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.091+0000 7fe3097fa700 1 -- 172.21.15.97:0/3092169019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31401c610 con 0x563cec5360e0 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- 172.21.15.97:0/3092169019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 msgr2=0x563cec536500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe314010fa0 tx=0x7fe314013000 comp rx=0 tx=0).stop 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- 172.21.15.97:0/3092169019 shutdown_connections 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec537d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec536500 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 --2- 172.21.15.97:0/3092169019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec5372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- 172.21.15.97:0/3092169019 >> 172.21.15.97:0/3092169019 conn(0x563cec4aa2e0 msgr2=0x563cec530e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- 172.21.15.97:0/3092169019 shutdown_connections 2023-10-08T09:56:30.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- 172.21.15.97:0/3092169019 wait complete. 2023-10-08T09:56:30.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.092+0000 7fe335742b80 1 -- start start 2023-10-08T09:56:30.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 Processor -- start 2023-10-08T09:56:30.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec547430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec488eb0 con 0x563cec536a40 2023-10-08T09:56:30.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec425790 con 0x563cec5360e0 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec425430 con 0x563cec537880 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe335742b80 10 client.0 _ll_register_callbacks cb 0x563cec282880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37498/0 (socket says 172.21.15.97:37498) 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe322ffd700 1 -- 172.21.15.97:0/1178785967 learned_addr learned my addr 172.21.15.97:0/1178785967 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3237fe700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec547430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 -- 172.21.15.97:0/1178785967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 msgr2=0x563cec547430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec547430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 -- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 msgr2=0x563cec53feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.093+0000 7fe3227fc700 1 -- 172.21.15.97:0/1178785967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe314010c50 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe3237fe700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec547430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe3227fc700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5403f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe31800e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559c4631b7193ede server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318011670 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe3180117d0 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cec450f30 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe318011990 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe335742b80 10 client.15855 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563cec29cce0 con 0x563cec536a40 2023-10-08T09:56:30.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cec450f30 con 0x563cec536a40 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe31800c070 con 0x563cec536a40 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.094+0000 7fe335742b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe335742b80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe335742b80 20 client.15855 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec54bc30 0x563cec54e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe335742b80 4 client.15855 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cec481e90 con 0x563cec54bc30 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe322ffd700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec54bc30 0x563cec54e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe306ff5700 10 client.15855 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.095+0000 7fe322ffd700 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec54bc30 0x563cec54e050 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8c828db36ddaf0 server_cookie=9f5c9a4df3bcf38e in_seq=0 out_seq=0 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe306ff5700 1 -- 172.21.15.97:0/1178785967 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563cec481e90 con 0x563cec54bc30 2023-10-08T09:56:30.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe306ff5700 10 client.15855 handle_command_reply: tid=0 2023-10-08T09:56:30.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 client.15855 shutdown 2023-10-08T09:56:30.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 msgr2=0x563cec5403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5403f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe31800e6a0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 shutdown_connections 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec54bc30 0x563cec54e050 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec547430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 --2- 172.21.15.97:0/1178785967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5403f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 >> 172.21.15.97:0/1178785967 conn(0x563cec4aa2e0 msgr2=0x563cec530550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe306ff5700 0 client.15855 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 shutdown_connections 2023-10-08T09:56:30.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.096+0000 7fe335742b80 1 -- 172.21.15.97:0/1178785967 wait complete. 2023-10-08T09:56:30.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.097+0000 7fe335742b80 20 client.15855 trim_cache size 0 max 16384 2023-10-08T09:56:30.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 Processor -- start 2023-10-08T09:56:30.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 -- start start 2023-10-08T09:56:30.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec53ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec3099a0 con 0x563cec536a40 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec52b5f0 con 0x563cec5360e0 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe335742b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec528b90 con 0x563cec537880 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57538/0 (socket says 172.21.15.97:57538) 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 -- 172.21.15.97:0/2372500314 learned_addr learned my addr 172.21.15.97:0/2372500314 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3237fe700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec53ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe322ffd700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 -- 172.21.15.97:0/2372500314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 msgr2=0x563cec53ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec53ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 -- 172.21.15.97:0/2372500314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 msgr2=0x563cec53a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3227fc700 1 -- 172.21.15.97:0/2372500314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cec52d320 con 0x563cec536a40 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.122+0000 7fe3237fe700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec53ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.123+0000 7fe3227fc700 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe318001eb0 tx=0x7fe318005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e27ab06a3fc7605 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.123+0000 7fe309ffb700 1 -- 172.21.15.97:0/2372500314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318010ec0 con 0x563cec536a40 2023-10-08T09:56:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.123+0000 7fe309ffb700 1 -- 172.21.15.97:0/2372500314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe318004500 con 0x563cec536a40 2023-10-08T09:56:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.123+0000 7fe309ffb700 1 -- 172.21.15.97:0/2372500314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31801a910 con 0x563cec536a40 2023-10-08T09:56:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- 172.21.15.97:0/2372500314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 msgr2=0x563cec5473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe318001eb0 tx=0x7fe318005f60 comp rx=0 tx=0).stop 2023-10-08T09:56:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- 172.21.15.97:0/2372500314 shutdown_connections 2023-10-08T09:56:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec537880 0x563cec53ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec5360e0 0x563cec53a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 --2- 172.21.15.97:0/2372500314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec536a40 0x563cec5473f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- 172.21.15.97:0/2372500314 >> 172.21.15.97:0/2372500314 conn(0x563cec4aa2e0 msgr2=0x563cec54dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- 172.21.15.97:0/2372500314 shutdown_connections 2023-10-08T09:56:30.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- 172.21.15.97:0/2372500314 wait complete. 2023-10-08T09:56:30.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 -- start start 2023-10-08T09:56:30.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.124+0000 7fe335742b80 1 Processor -- start 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec545ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec59fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec546c10 con 0x563cec5360e0 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec546d80 con 0x563cec537880 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cec5a0010 con 0x563cec536a40 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe335742b80 10 client.0 _ll_register_callbacks cb 0x7fe318009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59244/0 (socket says 172.21.15.97:59244) 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 -- 172.21.15.97:0/3557471712 learned_addr learned my addr 172.21.15.97:0/3557471712 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3237fe700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec59fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 -- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 msgr2=0x563cec59fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec59fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 -- 172.21.15.97:0/3557471712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 msgr2=0x563cec545ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec545ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3227fc700 1 -- 172.21.15.97:0/3557471712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cec29cce0 con 0x563cec536a40 2023-10-08T09:56:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.125+0000 7fe3237fe700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec59fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe3227fc700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe31800a010 tx=0x7fe318010bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ea27001da70cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe306ff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318033070 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe318000e40 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe314015070 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe318010ec0 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe335742b80 10 client.25321 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563cec2da2d0 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.126+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe314015070 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe318036030 con 0x563cec536a40 2023-10-08T09:56:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 10 client.25321 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 10 client.25321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 20 client.25321 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec5a1510 0x563cec5a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 4 client.25321 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563cec481bb0 con 0x563cec5a1510 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe322ffd700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec5a1510 0x563cec5a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe322ffd700 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec5a1510 0x563cec5a3930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e45b2bdea9e4c38 server_cookie=e5da326398df392b in_seq=0 out_seq=0 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.127+0000 7fe306ff5700 10 client.25321 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe306ff5700 1 -- 172.21.15.97:0/3557471712 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563cec481bb0 con 0x563cec5a1510 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe306ff5700 10 client.25321 handle_command_reply: tid=0 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 client.25321 shutdown 2023-10-08T09:56:30.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 msgr2=0x563cec546430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe31800a010 tx=0x7fe318010bc0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 shutdown_connections 2023-10-08T09:56:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563cec537880 0x563cec59fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cec5360e0 0x563cec545ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563cec5a1510 0x563cec5a3930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 --2- 172.21.15.97:0/3557471712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563cec536a40 0x563cec546430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 >> 172.21.15.97:0/3557471712 conn(0x563cec4aa2e0 msgr2=0x563cec53a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe306ff5700 0 client.25321 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 shutdown_connections 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.128+0000 7fe335742b80 1 -- 172.21.15.97:0/3557471712 wait complete. 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe335742b80 20 client.25321 trim_cache size 0 max 16384 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:56:30.158 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2023-10-08T09:56:30.159 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "db8ed3eb-c1ac-4bcc-b351-4f5cb99d14ff", 2023-10-08T09:56:30.159 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2023-10-08T09:56:30.159 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 msgr2=0x7fe30c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 0x7fe30c0a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe314005950 tx=0x7fe31400b410 comp rx=0 tx=0).stop 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 msgr2=0x7fe32419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007ab0 tx=0x7fe31c004c80 comp rx=0 tx=0).stop 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 shutdown_connections 2023-10-08T09:56:30.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe30c0a1480 0x7fe30c0a3940 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3241064f0 0x7fe32419ddb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe324101290 0x7fe3241968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 --2- 172.21.15.97:0/858298952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe324103bc0 0x7fe324196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 >> 172.21.15.97:0/858298952 conn(0x7fe3240fac40 msgr2=0x7fe324101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 shutdown_connections 2023-10-08T09:56:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.129+0000 7fe329559700 1 -- 172.21.15.97:0/858298952 wait complete. 2023-10-08T09:56:30.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:56:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/1302496882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 msgr2=0x7f4fc40ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/1302496882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc40ffac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00098f0 tx=0x7f4fc0009c00 comp rx=0 tx=0).stop 2023-10-08T09:56:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/1302496882 shutdown_connections 2023-10-08T09:56:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/1302496882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 0x7f4fc4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/1302496882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc40ffac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/1302496882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/1302496882 >> 172.21.15.97:0/1302496882 conn(0x7f4fc40fac10 msgr2=0x7f4fc40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/1302496882 shutdown_connections 2023-10-08T09:56:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.304+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/1302496882 wait complete. 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 Processor -- start 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 -- start start 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc4196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 0x7f4fc419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e2b0 con 0x7f4fc4100890 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e3f0 con 0x7f4fc4101230 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fcbeb4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc419e580 con 0x7f4fc40ff6a0 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57560/0 (socket says 172.21.15.97:57560) 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fc944f700 1 -- 172.21.15.97:0/3587211513 learned_addr learned my addr 172.21.15.97:0/3587211513 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.305+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc4196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fca451700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 0x7f4fc419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 -- 172.21.15.97:0/3587211513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 msgr2=0x7f4fc4196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc4196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 -- 172.21.15.97:0/3587211513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 msgr2=0x7f4fc419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 0x7f4fc419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 -- 172.21.15.97:0/3587211513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0009520 con 0x7f4fc4100890 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc4196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fc944f700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9c0 tx=0x7f4fb400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fbaffd700 1 -- 172.21.15.97:0/3587211513 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004a30 con 0x7f4fc4100890 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fbaffd700 1 -- 172.21.15.97:0/3587211513 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb400cc80 con 0x7f4fc4100890 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4108300 con 0x7f4fc4100890 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fbaffd700 1 -- 172.21.15.97:0/3587211513 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40054e0 con 0x7f4fc4100890 2023-10-08T09:56:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.306+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc4108810 con 0x7f4fc4100890 2023-10-08T09:56:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.309+0000 7f4fbaffd700 1 -- 172.21.15.97:0/3587211513 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4fb4004b90 con 0x7f4fc4100890 2023-10-08T09:56:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.310+0000 7f4fbaffd700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 0x7f4fb00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.310+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 0x7f4fb00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.310+0000 7f4fbaffd700 1 -- 172.21.15.97:0/3587211513 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fb4014070 con 0x7f4fc4100890 2023-10-08T09:56:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 0x7f4fb00a3940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009fd0 tx=0x7f4fc001d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 Processor -- start 2023-10-08T09:56:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 -- start start 2023-10-08T09:56:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb1501c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f84460 con 0x55bb1501d800 2023-10-08T09:56:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f75c20 con 0x55bb1501c060 2023-10-08T09:56:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.311+0000 7f4fd809db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f73540 con 0x55bb1501c9c0 2023-10-08T09:56:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fca451700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb1501c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59266/0 (socket says 172.21.15.97:59266) 2023-10-08T09:56:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fca451700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57568/0 (socket says 172.21.15.97:57568) 2023-10-08T09:56:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb1501c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37534/0 (socket says 172.21.15.97:37534) 2023-10-08T09:56:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 -- 172.21.15.97:0/257580260 learned_addr learned my addr 172.21.15.97:0/257580260 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 -- 172.21.15.97:0/257580260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 msgr2=0x55bb1501c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb1501c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 -- 172.21.15.97:0/257580260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 msgr2=0x55bb1501dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 -- 172.21.15.97:0/257580260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb14d99de0 con 0x55bb1501c9c0 2023-10-08T09:56:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fca451700 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:56:30.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4fc944f700 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a1850 tx=0x7f4fb4095e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c04c858c3e789435 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4faffff700 1 -- 172.21.15.97:0/257580260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400cdf0 con 0x55bb1501c9c0 2023-10-08T09:56:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4faffff700 1 -- 172.21.15.97:0/257580260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb409a0f0 con 0x55bb1501c9c0 2023-10-08T09:56:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.312+0000 7f4faffff700 1 -- 172.21.15.97:0/257580260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4005640 con 0x55bb1501c9c0 2023-10-08T09:56:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 -- 172.21.15.97:0/257580260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 msgr2=0x55bb1501d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a1850 tx=0x7f4fb4095e40 comp rx=0 tx=0).stop 2023-10-08T09:56:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 -- 172.21.15.97:0/257580260 shutdown_connections 2023-10-08T09:56:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501d800 0x55bb1501dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb1501c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 --2- 172.21.15.97:0/257580260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1501d270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 -- 172.21.15.97:0/257580260 >> 172.21.15.97:0/257580260 conn(0x55bb14f90260 msgr2=0x55bb15016e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 -- 172.21.15.97:0/257580260 shutdown_connections 2023-10-08T09:56:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.313+0000 7f4fd809db80 1 -- 172.21.15.97:0/257580260 wait complete. 2023-10-08T09:56:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 -- start start 2023-10-08T09:56:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 Processor -- start 2023-10-08T09:56:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15026460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501d800 0x55bb1502d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f6ee30 con 0x55bb1501c9c0 2023-10-08T09:56:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f0b710 con 0x55bb1501c060 2023-10-08T09:56:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f0b3b0 con 0x55bb1501d800 2023-10-08T09:56:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fd809db80 10 client.0 _ll_register_callbacks cb 0x55bb14d68800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37540/0 (socket says 172.21.15.97:37540) 2023-10-08T09:56:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.314+0000 7f4fc9c50700 1 -- 172.21.15.97:0/1128526810 learned_addr learned my addr 172.21.15.97:0/1128526810 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc944f700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15026460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fca451700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501d800 0x55bb1502d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 -- 172.21.15.97:0/1128526810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501d800 msgr2=0x55bb1502d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501d800 0x55bb1502d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 -- 172.21.15.97:0/1128526810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 msgr2=0x55bb15026460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15026460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 -- 172.21.15.97:0/1128526810 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc011230 con 0x55bb1501c060 2023-10-08T09:56:30.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000fb70 tx=0x7f4fc0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7716438ad102b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-08T09:56:30.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000bbc0 con 0x55bb1501c060 2023-10-08T09:56:30.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb14f36eb0 con 0x55bb1501c060 2023-10-08T09:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fc0005040 con 0x55bb1501c060 2023-10-08T09:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc000f820 con 0x55bb1501c060 2023-10-08T09:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fd809db80 10 client.25316 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:30.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.315+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f4fc4108810 con 0x55bb1501c060 2023-10-08T09:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb14f36eb0 con 0x55bb1501c060 2023-10-08T09:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fc0003d00 con 0x55bb1501c060 2023-10-08T09:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 10 client.25316 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:30.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 20 client.25316 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb15031c50 0x55bb15034070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 4 client.25316 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb14f67e10 con 0x55bb15031c50 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.316+0000 7f4fc944f700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb15031c50 0x55bb15034070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fad7fa700 10 client.25316 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fc944f700 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb15031c50 0x55bb15034070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d06dcc2c6f4ad54 server_cookie=868fc12375655455 in_seq=0 out_seq=0 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fad7fa700 1 -- 172.21.15.97:0/1128526810 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb14f67e10 con 0x55bb15031c50 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fad7fa700 10 client.25316 handle_command_reply: tid=0 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fd809db80 1 client.25316 shutdown 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 msgr2=0x55bb15025f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.317+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000fb70 tx=0x7f4fc0009260 comp rx=0 tx=0).stop 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 shutdown_connections 2023-10-08T09:56:30.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb15031c50 0x55bb15034070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501d800 0x55bb1502d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fad7fa700 0 client.25316 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501c060 0x55bb15025f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 --2- 172.21.15.97:0/1128526810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15026460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 >> 172.21.15.97:0/1128526810 conn(0x55bb14f90260 msgr2=0x55bb150164d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 shutdown_connections 2023-10-08T09:56:30.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 1 -- 172.21.15.97:0/1128526810 wait complete. 2023-10-08T09:56:30.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.318+0000 7f4fd809db80 20 client.25316 trim_cache size 0 max 16384 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 Processor -- start 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 -- start start 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1502d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb15020860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14def920 con 0x55bb1501c060 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb14f36eb0 con 0x55bb1501d800 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fd809db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb1502e5d0 con 0x55bb1501c9c0 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fc9c50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57578/0 (socket says 172.21.15.97:57578) 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fc9c50700 1 -- 172.21.15.97:0/638465222 learned_addr learned my addr 172.21.15.97:0/638465222 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.340+0000 7f4fca451700 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb15020860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 -- 172.21.15.97:0/638465222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 msgr2=0x55bb1502d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1502d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 -- 172.21.15.97:0/638465222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 msgr2=0x55bb15020860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb15020860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 -- 172.21.15.97:0/638465222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb14d82c60 con 0x55bb1501c060 2023-10-08T09:56:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fca451700 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb15020860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0032490 tx=0x7f4fc00059a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e60a7695dd7925 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fb8ff9700 1 -- 172.21.15.97:0/638465222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000ba40 con 0x55bb1501c060 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fb8ff9700 1 -- 172.21.15.97:0/638465222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc0021040 con 0x55bb1501c060 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.341+0000 7f4fb8ff9700 1 -- 172.21.15.97:0/638465222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0032b40 con 0x55bb1501c060 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- 172.21.15.97:0/638465222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 msgr2=0x55bb15025430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0032490 tx=0x7f4fc00059a0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- 172.21.15.97:0/638465222 shutdown_connections 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb15020860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c060 0x55bb15025430 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 --2- 172.21.15.97:0/638465222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c9c0 0x55bb1502d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- 172.21.15.97:0/638465222 >> 172.21.15.97:0/638465222 conn(0x55bb14f90260 msgr2=0x55bb15012f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- 172.21.15.97:0/638465222 shutdown_connections 2023-10-08T09:56:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- 172.21.15.97:0/638465222 wait complete. 2023-10-08T09:56:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 -- start start 2023-10-08T09:56:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.342+0000 7f4fd809db80 1 Processor -- start 2023-10-08T09:56:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 0x55bb1502d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb1502a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb1502a5a0 con 0x55bb1501c9c0 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb1502a710 con 0x55bb1501d800 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15096a80 con 0x55bb1501c060 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 10 client.0 _ll_register_callbacks cb 0x7f4fc001bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57586/0 (socket says 172.21.15.97:57586) 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 -- 172.21.15.97:0/2150934242 learned_addr learned my addr 172.21.15.97:0/2150934242 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 0x55bb1502d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 -- 172.21.15.97:0/2150934242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 msgr2=0x55bb1502d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 0x55bb1502d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 -- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 msgr2=0x55bb1502a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb1502a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 -- 172.21.15.97:0/2150934242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc4108810 con 0x55bb1501c9c0 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 0x55bb1502d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fc944f700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005950 tx=0x7f4fb4099b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d46ec5721923a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:30.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2210 con 0x55bb1501c9c0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fb409db40 con 0x55bb1501c9c0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.343+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fb409a0f0 con 0x55bb1501c9c0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fb40652b0 con 0x55bb1501c9c0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 10 client.15876 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb14dc0250 con 0x55bb1501c9c0 2023-10-08T09:56:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fb407e080 con 0x55bb1501c9c0 2023-10-08T09:56:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fb409ad20 con 0x55bb1501c9c0 2023-10-08T09:56:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 20 client.15876 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb150981f0 0x55bb1509a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 4 client.15876 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb14f67b30 con 0x55bb150981f0 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.344+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb150981f0 0x55bb1509a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fc9c50700 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb150981f0 0x55bb1509a610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13d053eb555e6ae server_cookie=fc862fb62938036b in_seq=0 out_seq=0 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fad7fa700 10 client.15876 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fad7fa700 1 -- 172.21.15.97:0/2150934242 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bb14f67b30 con 0x55bb150981f0 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fad7fa700 10 client.15876 handle_command_reply: tid=0 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 client.15876 shutdown 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 msgr2=0x55bb15029b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005950 tx=0x7f4fb4099b10 comp rx=0 tx=0).stop 2023-10-08T09:56:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 shutdown_connections 2023-10-08T09:56:30.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb1501d800 0x55bb1502a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55bb150981f0 0x55bb1509a610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bb1501c060 0x55bb1502d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 --2- 172.21.15.97:0/2150934242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb1501c9c0 0x55bb15029b20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fad7fa700 0 client.15876 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:30.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 >> 172.21.15.97:0/2150934242 conn(0x55bb14f90260 msgr2=0x55bb15025430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 shutdown_connections 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 1 -- 172.21.15.97:0/2150934242 wait complete. 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.345+0000 7f4fd809db80 20 client.15876 trim_cache size 0 max 16384 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2023-10-08T09:56:30.367 INFO:teuthology.orchestra.run.smithi097.stdout: "db8ed3eb-c1ac-4bcc-b351-4f5cb99d14ff": { 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "db8ed3eb-c1ac-4bcc-b351-4f5cb99d14ff", 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:56:30.368 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:56:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 msgr2=0x7f4fb00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 0x7f4fb00a3940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009fd0 tx=0x7f4fc001d040 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 msgr2=0x7f4fc4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9c0 tx=0x7f4fb400ecd0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 shutdown_connections 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4fb00a1480 0x7f4fb00a3940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fc4101230 0x7f4fc419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc40ff6a0 0x7f4fc4196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 --2- 172.21.15.97:0/3587211513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc4100890 0x7f4fc4196cc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 >> 172.21.15.97:0/3587211513 conn(0x7f4fc40fac10 msgr2=0x7f4fc4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 shutdown_connections 2023-10-08T09:56:30.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:56:30.346+0000 7f4fcbeb4700 1 -- 172.21.15.97:0/3587211513 wait complete. 2023-10-08T09:56:30.371 INFO:tasks.cephfs.filesystem:scrub status for tag:db8ed3eb-c1ac-4bcc-b351-4f5cb99d14ff - {'path': '/', 'tag': 'db8ed3eb-c1ac-4bcc-b351-4f5cb99d14ff', 'options': 'recursive,force'} 2023-10-08T09:56:31.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: pgmap v341: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:31.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: scrub queued for path: / 2023-10-08T09:56:31.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:56:31.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:31 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: pgmap v341: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: scrub queued for path: / 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:31 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: pgmap v341: 129 pgs: 129 active+clean; 588 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: scrub queued for path: / 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:56:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:31 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T09:56:33.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:33 smithi097 ceph-mon[99491]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2023-10-08T09:56:33.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:33 smithi178 ceph-mon[103156]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2023-10-08T09:56:33.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:33 smithi186 ceph-mon[104393]: pgmap v342: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2023-10-08T09:56:35.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:35 smithi097 ceph-mon[99491]: pgmap v343: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2023-10-08T09:56:35.465 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:35 smithi178 ceph-mon[103156]: pgmap v343: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2023-10-08T09:56:35.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:35 smithi186 ceph-mon[104393]: pgmap v343: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2023-10-08T09:56:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 -- 172.21.15.97:0/1509751302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 msgr2=0x7faf74107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 --2- 172.21.15.97:0/1509751302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74107b70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b0a0 tx=0x7faf6c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 -- 172.21.15.97:0/1509751302 shutdown_connections 2023-10-08T09:56:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 --2- 172.21.15.97:0/1509751302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74107b70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 --2- 172.21.15.97:0/1509751302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 0x7faf74101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 --2- 172.21.15.97:0/1509751302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 0x7faf740ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 -- 172.21.15.97:0/1509751302 >> 172.21.15.97:0/1509751302 conn(0x7faf740fa340 msgr2=0x7faf740fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.869+0000 7faf7b809700 1 -- 172.21.15.97:0/1509751302 shutdown_connections 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 -- 172.21.15.97:0/1509751302 wait complete. 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 Processor -- start 2023-10-08T09:56:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 -- start start 2023-10-08T09:56:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 0x7faf74195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 0x7faf7419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419d1b0 con 0x7faf741002b0 2023-10-08T09:56:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419d2f0 con 0x7faf740ff910 2023-10-08T09:56:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.870+0000 7faf7b809700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7419d480 con 0x7faf74100be0 2023-10-08T09:56:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57600/0 (socket says 172.21.15.97:57600) 2023-10-08T09:56:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 -- 172.21.15.97:0/1324127533 learned_addr learned my addr 172.21.15.97:0/1324127533 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf795a5700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 0x7faf74195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf79da6700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 0x7faf7419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 -- 172.21.15.97:0/1324127533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 msgr2=0x7faf7419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 0x7faf7419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 -- 172.21.15.97:0/1324127533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 msgr2=0x7faf74195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 0x7faf74195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 -- 172.21.15.97:0/1324127533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009d00 con 0x7faf741002b0 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf78da4700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf6400d980 tx=0x7faf6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf6a7fc700 1 -- 172.21.15.97:0/1324127533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64004220 con 0x7faf741002b0 2023-10-08T09:56:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.871+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740fe920 con 0x7faf741002b0 2023-10-08T09:56:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.872+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740fee80 con 0x7faf741002b0 2023-10-08T09:56:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.872+0000 7faf6a7fc700 1 -- 172.21.15.97:0/1324127533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf64007640 con 0x7faf741002b0 2023-10-08T09:56:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.872+0000 7faf6a7fc700 1 -- 172.21.15.97:0/1324127533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64005610 con 0x7faf741002b0 2023-10-08T09:56:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.875+0000 7faf6a7fc700 1 -- 172.21.15.97:0/1324127533 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faf64004380 con 0x7faf741002b0 2023-10-08T09:56:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.875+0000 7faf6a7fc700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 0x7faf600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.876+0000 7faf795a5700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 0x7faf600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.876+0000 7faf6a7fc700 1 -- 172.21.15.97:0/1324127533 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faf640cce00 con 0x7faf741002b0 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.876+0000 7faf795a5700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 0x7faf600a39c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf74067840 tx=0x7faf7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 Processor -- start 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 -- start start 2023-10-08T09:56:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da50c400 con 0x55d8da5a47a0 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da4fdbc0 con 0x55d8da5a52b0 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf879f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da4fb4e0 con 0x55d8da5a3e40 2023-10-08T09:56:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf78da4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf78da4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57608/0 (socket says 172.21.15.97:57608) 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf78da4700 1 -- 172.21.15.97:0/3728903423 learned_addr learned my addr 172.21.15.97:0/3728903423 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.878+0000 7faf795a5700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf79da6700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 -- 172.21.15.97:0/3728903423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 msgr2=0x55d8da5a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 -- 172.21.15.97:0/3728903423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 msgr2=0x55d8da5a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 -- 172.21.15.97:0/3728903423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8da321d80 con 0x55d8da5a47a0 2023-10-08T09:56:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf79da6700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:56:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf78da4700 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf640a1c30 tx=0x7faf64099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b65a121ccd60c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf5f7fe700 1 -- 172.21.15.97:0/3728903423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640a2160 con 0x55d8da5a47a0 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf5f7fe700 1 -- 172.21.15.97:0/3728903423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf64022e10 con 0x55d8da5a47a0 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.879+0000 7faf5f7fe700 1 -- 172.21.15.97:0/3728903423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640a1f30 con 0x55d8da5a47a0 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.880+0000 7faf879f2b80 1 -- 172.21.15.97:0/3728903423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 msgr2=0x55d8da5a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.880+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faf640a1c30 tx=0x7faf64099860 comp rx=0 tx=0).stop 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 -- 172.21.15.97:0/3728903423 shutdown_connections 2023-10-08T09:56:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3728903423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a47a0 0x55d8da5a4cd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 -- 172.21.15.97:0/3728903423 >> 172.21.15.97:0/3728903423 conn(0x55d8da518200 msgr2=0x55d8da59e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 -- 172.21.15.97:0/3728903423 shutdown_connections 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 -- 172.21.15.97:0/3728903423 wait complete. 2023-10-08T09:56:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 -- start start 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.881+0000 7faf879f2b80 1 Processor -- start 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a3e40 0x55d8da5ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da4f6dd0 con 0x55d8da5a52b0 2023-10-08T09:56:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da4936b0 con 0x55d8da5a3e40 2023-10-08T09:56:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da493350 con 0x55d8da5a47a0 2023-10-08T09:56:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf879f2b80 10 client.0 _ll_register_callbacks cb 0x55d8da2f07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57614/0 (socket says 172.21.15.97:57614) 2023-10-08T09:56:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 -- 172.21.15.97:0/1972308772 learned_addr learned my addr 172.21.15.97:0/1972308772 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf795a5700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a3e40 0x55d8da5ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 -- 172.21.15.97:0/1972308772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 msgr2=0x55d8da5ade40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5ade40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 -- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a3e40 msgr2=0x55d8da5ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a3e40 0x55d8da5ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.882+0000 7faf79da6700 1 -- 172.21.15.97:0/1972308772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740fee80 con 0x55d8da5a52b0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf79da6700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b830 tx=0x7faf6c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350d9b2fcc592598 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00e040 con 0x55d8da5a52b0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faf6c01c440 con 0x55d8da5a52b0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8da4bee50 con 0x55d8da5a52b0 2023-10-08T09:56:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf6c009b80 con 0x55d8da5a52b0 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf879f2b80 10 client.15888 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d8da30ac00 con 0x55d8da5a52b0 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.883+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8da4bee50 con 0x55d8da5a52b0 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faf6c019050 con 0x55d8da5a52b0 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 20 client.15888 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da5b9660 0x55d8da5bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 4 client.15888 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8da4efdb0 con 0x55d8da5b9660 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf795a5700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da5b9660 0x55d8da5bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf5cff9700 10 client.15888 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.884+0000 7faf795a5700 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da5b9660 0x55d8da5bba80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba8dea308bf7acb server_cookie=d403c6fc37ee2474 in_seq=0 out_seq=0 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf5cff9700 1 -- 172.21.15.97:0/1972308772 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8da4efdb0 con 0x55d8da5b9660 2023-10-08T09:56:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf5cff9700 10 client.15888 handle_command_reply: tid=0 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 client.15888 shutdown 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 msgr2=0x55d8da5b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b830 tx=0x7faf6c009300 comp rx=0 tx=0).stop 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 shutdown_connections 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b4de0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a3e40 0x55d8da5ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da5b9660 0x55d8da5bba80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 --2- 172.21.15.97:0/1972308772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 >> 172.21.15.97:0/1972308772 conn(0x55d8da518200 msgr2=0x55d8da59e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.885+0000 7faf5cff9700 0 client.15888 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.886+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 shutdown_connections 2023-10-08T09:56:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.886+0000 7faf879f2b80 1 -- 172.21.15.97:0/1972308772 wait complete. 2023-10-08T09:56:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.886+0000 7faf879f2b80 20 client.15888 trim_cache size 0 max 16384 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 Processor -- start 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 -- start start 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da3778c0 con 0x55d8da5a3e40 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da59b860 con 0x55d8da5a52b0 2023-10-08T09:56:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf879f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da5ad470 con 0x55d8da5a47a0 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf795a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf795a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57618/0 (socket says 172.21.15.97:57618) 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.914+0000 7faf795a5700 1 -- 172.21.15.97:0/3964068155 learned_addr learned my addr 172.21.15.97:0/3964068155 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf79da6700 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 -- 172.21.15.97:0/3964068155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 msgr2=0x55d8da5b4da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5b4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 -- 172.21.15.97:0/3964068155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 msgr2=0x55d8da5ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 -- 172.21.15.97:0/3964068155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8da3481f0 con 0x55d8da5a3e40 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf795a5700 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55d8da5bc360 tx=0x7faf70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fa94e7b6a52bf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf5ffff700 1 -- 172.21.15.97:0/3964068155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7002b070 con 0x55d8da5a3e40 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf5ffff700 1 -- 172.21.15.97:0/3964068155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf70015070 con 0x55d8da5a3e40 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.915+0000 7faf5ffff700 1 -- 172.21.15.97:0/3964068155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70025070 con 0x55d8da5a3e40 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 -- 172.21.15.97:0/3964068155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 msgr2=0x55d8da5b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55d8da5bc360 tx=0x7faf70013000 comp rx=0 tx=0).stop 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 -- 172.21.15.97:0/3964068155 shutdown_connections 2023-10-08T09:56:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a52b0 0x55d8da5ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a3e40 0x55d8da5b5490 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 --2- 172.21.15.97:0/3964068155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a47a0 0x55d8da5b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 -- 172.21.15.97:0/3964068155 >> 172.21.15.97:0/3964068155 conn(0x55d8da518200 msgr2=0x55d8da5c03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 -- 172.21.15.97:0/3964068155 shutdown_connections 2023-10-08T09:56:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.916+0000 7faf879f2b80 1 -- 172.21.15.97:0/3964068155 wait complete. 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 -- start start 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 Processor -- start 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a47a0 0x55d8da5b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da5b43d0 con 0x55d8da5a52b0 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da5b4540 con 0x55d8da5a47a0 2023-10-08T09:56:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8da5b4700 con 0x55d8da5a3e40 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.917+0000 7faf879f2b80 10 client.0 _ll_register_callbacks cb 0x7faf6c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59324/0 (socket says 172.21.15.97:59324) 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 -- 172.21.15.97:0/799209148 learned_addr learned my addr 172.21.15.97:0/799209148 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf79da6700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf78da4700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a47a0 0x55d8da5b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 -- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a47a0 msgr2=0x55d8da5b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a47a0 0x55d8da5b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 -- 172.21.15.97:0/799209148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 msgr2=0x55d8da5b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 -- 172.21.15.97:0/799209148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf70010a20 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf795a5700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf7000b950 tx=0x7faf70004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce564810cbd4ec0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70025030 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf64022e10 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.918+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faf70017070 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf7002b070 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 10 client.25345 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d8da320250 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf64022e10 con 0x55d8da5a3e40 2023-10-08T09:56:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faf70011710 con 0x55d8da5a3e40 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 10 client.25345 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 20 client.25345 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da60dd40 0x55d8da610160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 4 client.25345 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8da4efad0 con 0x55d8da60dd40 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.919+0000 7faf78da4700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da60dd40 0x55d8da610160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.920+0000 7faf5cff9700 10 client.25345 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.920+0000 7faf78da4700 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da60dd40 0x55d8da610160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5de67d8617095d3 server_cookie=332e433977c692eb in_seq=0 out_seq=0 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.926+0000 7faf5cff9700 1 -- 172.21.15.97:0/799209148 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8da4efad0 con 0x55d8da60dd40 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.926+0000 7faf5cff9700 10 client.25345 handle_command_reply: tid=0 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.926+0000 7faf879f2b80 1 client.25345 shutdown 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 msgr2=0x55d8da5b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf7000b950 tx=0x7faf70004490 comp rx=0 tx=0).stop 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 shutdown_connections 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8da5a52b0 0x55d8da5b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55d8da60dd40 0x55d8da610160 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8da5a3e40 0x55d8da5b2990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf5cff9700 0 client.25345 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 --2- 172.21.15.97:0/799209148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8da5a47a0 0x55d8da5b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 >> 172.21.15.97:0/799209148 conn(0x55d8da518200 msgr2=0x55d8da5ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 shutdown_connections 2023-10-08T09:56:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 1 -- 172.21.15.97:0/799209148 wait complete. 2023-10-08T09:56:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf879f2b80 20 client.25345 trim_cache size 0 max 16384 2023-10-08T09:56:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:56:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:56:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:56:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 msgr2=0x7faf600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 0x7faf600a39c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf74067840 tx=0x7faf7000b410 comp rx=0 tx=0).stop 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 msgr2=0x7faf74195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.927+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf6400d980 tx=0x7faf6400dc90 comp rx=0 tx=0).stop 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 shutdown_connections 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7faf600a1500 0x7faf600a39c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf74100be0 0x7faf7419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf740ff910 0x7faf74195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 --2- 172.21.15.97:0/1324127533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf741002b0 0x7faf74195bc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 >> 172.21.15.97:0/1324127533 conn(0x7faf740fa340 msgr2=0x7faf74103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 shutdown_connections 2023-10-08T09:56:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:36.928+0000 7faf7b809700 1 -- 172.21.15.97:0/1324127533 wait complete. 2023-10-08T09:56:37.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:37 smithi178 ceph-mon[103156]: pgmap v344: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-08T09:56:37.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:37 smithi186 ceph-mon[104393]: pgmap v344: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-08T09:56:37.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:37 smithi097 ceph-mon[99491]: pgmap v344: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-08T09:56:39.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:39 smithi186 ceph-mon[104393]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:39.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:39 smithi097 ceph-mon[99491]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:39 smithi178 ceph-mon[103156]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-10-08T09:56:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:41 smithi186 ceph-mon[104393]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T09:56:41.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:41 smithi097 ceph-mon[99491]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T09:56:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:41 smithi178 ceph-mon[103156]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T09:56:43.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:43 smithi186 ceph-mon[104393]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-08T09:56:43.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:43 smithi097 ceph-mon[99491]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-08T09:56:43.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:43 smithi178 ceph-mon[103156]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-08T09:56:45.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:45 smithi186 ceph-mon[104393]: pgmap v348: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-08T09:56:45.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:45 smithi097 ceph-mon[99491]: pgmap v348: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-08T09:56:45.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:45 smithi178 ceph-mon[103156]: pgmap v348: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-08T09:56:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:47 smithi186 ceph-mon[104393]: pgmap v349: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:47.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:47 smithi097 ceph-mon[99491]: pgmap v349: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:47 smithi178 ceph-mon[103156]: pgmap v349: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:49.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:49 smithi186 ceph-mon[104393]: pgmap v350: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:49.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:49 smithi097 ceph-mon[99491]: pgmap v350: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:49 smithi178 ceph-mon[103156]: pgmap v350: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-08T09:56:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 -- 172.21.15.97:0/534448797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f81011e0 msgr2=0x7fe7f8107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 --2- 172.21.15.97:0/534448797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f81011e0 0x7fe7f8107a90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c170 tx=0x7fe7f0009520 comp rx=0 tx=0).stop 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 -- 172.21.15.97:0/534448797 shutdown_connections 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 --2- 172.21.15.97:0/534448797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f81011e0 0x7fe7f8107a90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 --2- 172.21.15.97:0/534448797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f80ff650 0x7fe7f80ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 --2- 172.21.15.97:0/534448797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f8100840 0x7fe7f8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 -- 172.21.15.97:0/534448797 >> 172.21.15.97:0/534448797 conn(0x7fe7f80fabe0 msgr2=0x7fe7f80fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 -- 172.21.15.97:0/534448797 shutdown_connections 2023-10-08T09:56:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.094+0000 7fe7fd076700 1 -- 172.21.15.97:0/534448797 wait complete. 2023-10-08T09:56:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.095+0000 7fe7fd076700 1 Processor -- start 2023-10-08T09:56:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.095+0000 7fe7fd076700 1 -- start start 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f8100840 0x7fe7f8194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f81011e0 0x7fe7f819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819bf50 con 0x7fe7f80ff650 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819c090 con 0x7fe7f81011e0 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7fd076700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819c220 con 0x7fe7f8100840 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57638/0 (socket says 172.21.15.97:57638) 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/4075665593 learned_addr learned my addr 172.21.15.97:0/4075665593 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f759e700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f81011e0 0x7fe7f819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/4075665593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f8100840 msgr2=0x7fe7f8194960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f8100840 0x7fe7f8194960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/4075665593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f81011e0 msgr2=0x7fe7f819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f81011e0 0x7fe7f819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.096+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/4075665593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f000bd20 con 0x7fe7f80ff650 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800da70 tx=0x7fe7e800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7dffff700 1 -- 172.21.15.97:0/4075665593 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e800c9d0 con 0x7fe7f80ff650 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f819c500 con 0x7fe7f80ff650 2023-10-08T09:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7dffff700 1 -- 172.21.15.97:0/4075665593 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e8004020 con 0x7fe7f80ff650 2023-10-08T09:56:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7dffff700 1 -- 172.21.15.97:0/4075665593 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003cc0 con 0x7fe7f80ff650 2023-10-08T09:56:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.097+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f819ca60 con 0x7fe7f80ff650 2023-10-08T09:56:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.100+0000 7fe7dffff700 1 -- 172.21.15.97:0/4075665593 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe7e8004190 con 0x7fe7f80ff650 2023-10-08T09:56:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.101+0000 7fe7dffff700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 0x7fe7e00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.101+0000 7fe7f659c700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 0x7fe7e00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.101+0000 7fe7dffff700 1 -- 172.21.15.97:0/4075665593 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7e80cc6f0 con 0x7fe7f80ff650 2023-10-08T09:56:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.101+0000 7fe7f659c700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 0x7fe7e00a3940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec011fd0 tx=0x7fe7ec013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 Processor -- start 2023-10-08T09:56:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 -- start start 2023-10-08T09:56:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f887a0 0x5597b1f88cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1ef0400 con 0x5597b1f892b0 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1ee1bc0 con 0x5597b1f887a0 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.102+0000 7fe80925fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1edf4e0 con 0x5597b1f87e40 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57648/0 (socket says 172.21.15.97:57648) 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 -- 172.21.15.97:0/2334289139 learned_addr learned my addr 172.21.15.97:0/2334289139 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 -- 172.21.15.97:0/2334289139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 msgr2=0x5597b1f88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 -- 172.21.15.97:0/2334289139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f887a0 msgr2=0x5597b1f88cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f887a0 0x5597b1f88cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 -- 172.21.15.97:0/2334289139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597b1d05d80 con 0x5597b1f892b0 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7f759e700 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c170 tx=0x7fe7f0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fa0d302f0c456 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7dd7fa700 1 -- 172.21.15.97:0/2334289139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00096f0 con 0x5597b1f892b0 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7dd7fa700 1 -- 172.21.15.97:0/2334289139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f0003a00 con 0x5597b1f892b0 2023-10-08T09:56:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.103+0000 7fe7dd7fa700 1 -- 172.21.15.97:0/2334289139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000b4a0 con 0x5597b1f892b0 2023-10-08T09:56:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 -- 172.21.15.97:0/2334289139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 msgr2=0x5597b1f89730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c170 tx=0x7fe7f0006000 comp rx=0 tx=0).stop 2023-10-08T09:56:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 -- 172.21.15.97:0/2334289139 shutdown_connections 2023-10-08T09:56:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f89730 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 --2- 172.21.15.97:0/2334289139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f887a0 0x5597b1f88cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 -- 172.21.15.97:0/2334289139 >> 172.21.15.97:0/2334289139 conn(0x5597b1efc200 msgr2=0x5597b1f828a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.104+0000 7fe80925fb80 1 -- 172.21.15.97:0/2334289139 shutdown_connections 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 -- 172.21.15.97:0/2334289139 wait complete. 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 -- start start 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 Processor -- start 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f87e40 0x5597b1f91880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f91dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1edadd0 con 0x5597b1f892b0 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1e776b0 con 0x5597b1f87e40 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1e77350 con 0x5597b1f887a0 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe80925fb80 10 client.0 _ll_register_callbacks cb 0x5597b1cd47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe7f759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe7f759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57654/0 (socket says 172.21.15.97:57654) 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.105+0000 7fe7f759e700 1 -- 172.21.15.97:0/3562252669 learned_addr learned my addr 172.21.15.97:0/3562252669 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f87e40 0x5597b1f91880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 -- 172.21.15.97:0/3562252669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 msgr2=0x5597b1f91dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f91dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 -- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f87e40 msgr2=0x5597b1f91880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f87e40 0x5597b1f91880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 -- 172.21.15.97:0/3562252669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f0018040 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7f759e700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c140 tx=0x7fe7f00098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5620e5713aecc641 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7daff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0052070 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7f000ca60 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.106+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597b1ea2e50 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.107+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f000cc20 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.107+0000 7fe80925fb80 10 client.15906 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.107+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597b1ceec00 con 0x5597b1f892b0 2023-10-08T09:56:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.107+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597b1ea2e50 con 0x5597b1f892b0 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7f0055030 con 0x5597b1f892b0 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 20 client.15906 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1f9b990 0x5597b1f9ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 4 client.15906 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597b1ed3db0 con 0x5597b1f9b990 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.108+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1f9b990 0x5597b1f9ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.109+0000 7fe7daff5700 10 client.15906 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.109+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1f9b990 0x5597b1f9ddb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d03b55ea8e42501 server_cookie=1cdfee42efa787a3 in_seq=0 out_seq=0 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.109+0000 7fe7daff5700 1 -- 172.21.15.97:0/3562252669 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5597b1ed3db0 con 0x5597b1f9b990 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.109+0000 7fe7daff5700 10 client.15906 handle_command_reply: tid=0 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 client.15906 shutdown 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 msgr2=0x5597b1f98db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000c140 tx=0x7fe7f00098b0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 shutdown_connections 2023-10-08T09:56:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f892b0 0x5597b1f98db0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f87e40 0x5597b1f91880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1f9b990 0x5597b1f9ddb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 --2- 172.21.15.97:0/3562252669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f91dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 >> 172.21.15.97:0/3562252669 conn(0x5597b1efc200 msgr2=0x5597b1f82010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe7daff5700 0 client.15906 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 shutdown_connections 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 1 -- 172.21.15.97:0/3562252669 wait complete. 2023-10-08T09:56:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.110+0000 7fe80925fb80 20 client.15906 trim_cache size 0 max 16384 2023-10-08T09:56:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.137+0000 7fe80925fb80 1 Processor -- start 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.137+0000 7fe80925fb80 1 -- start start 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1f90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1d5b8c0 con 0x5597b1f887a0 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1f846e0 con 0x5597b1f892b0 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe80925fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1f83390 con 0x5597b1f87e40 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57662/0 (socket says 172.21.15.97:57662) 2023-10-08T09:56:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 -- 172.21.15.97:0/260677766 learned_addr learned my addr 172.21.15.97:0/260677766 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f759e700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1f90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 -- 172.21.15.97:0/260677766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 msgr2=0x5597b1f88240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 -- 172.21.15.97:0/260677766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 msgr2=0x5597b1f90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1f90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f659c700 1 -- 172.21.15.97:0/260677766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597b1d2c1f0 con 0x5597b1f887a0 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.138+0000 7fe7f759e700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1f90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.139+0000 7fe7f659c700 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec011fa0 tx=0x7fe7ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7086c5237e156e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.139+0000 7fe7ddffb700 1 -- 172.21.15.97:0/260677766 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec01dde0 con 0x5597b1f887a0 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.139+0000 7fe7ddffb700 1 -- 172.21.15.97:0/260677766 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ec017040 con 0x5597b1f887a0 2023-10-08T09:56:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.139+0000 7fe7ddffb700 1 -- 172.21.15.97:0/260677766 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec01b720 con 0x5597b1f887a0 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- 172.21.15.97:0/260677766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 msgr2=0x5597b1f98d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec011fa0 tx=0x7fe7ec004750 comp rx=0 tx=0).stop 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- 172.21.15.97:0/260677766 shutdown_connections 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1f90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f87e40 0x5597b1f88240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 --2- 172.21.15.97:0/260677766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f887a0 0x5597b1f98d70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- 172.21.15.97:0/260677766 >> 172.21.15.97:0/260677766 conn(0x5597b1efc200 msgr2=0x5597b1f7ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- 172.21.15.97:0/260677766 shutdown_connections 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- 172.21.15.97:0/260677766 wait complete. 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.140+0000 7fe80925fb80 1 -- start start 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 Processor -- start 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1fefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1f96890 con 0x5597b1f87e40 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1ff02b0 con 0x5597b1f892b0 2023-10-08T09:56:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b1ff03f0 con 0x5597b1f887a0 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe80925fb80 10 client.0 _ll_register_callbacks cb 0x7fe7f000bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57666/0 (socket says 172.21.15.97:57666) 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/1790742689 learned_addr learned my addr 172.21.15.97:0/1790742689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f759e700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1fefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/1790742689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 msgr2=0x5597b1f96240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f96240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 msgr2=0x5597b1fefd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1fefd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.141+0000 7fe7f6d9d700 1 -- 172.21.15.97:0/1790742689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec016de0 con 0x5597b1f87e40 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7f6d9d700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8095be0 tx=0x7fe7e80b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e15dd9dc2bafeb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e809d830 con 0x5597b1f87e40 2023-10-08T09:56:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7e80a1a70 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7f0003a00 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e8004470 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe80925fb80 10 client.15912 fetch_fsmap learned FSMap version 24 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597b1d04250 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f0003a00 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.142+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7e80b73e0 con 0x5597b1f87e40 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 20 client.15912 populate_metadata read hostname 'smithi097' 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1ff1e70 0x5597b1ff4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 4 client.15912 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:56:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.143+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597b1ed3ad0 con 0x5597b1ff1e70 2023-10-08T09:56:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.149+0000 7fe7f659c700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1ff1e70 0x5597b1ff4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:56:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.149+0000 7fe7daff5700 10 client.15912 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.149+0000 7fe7f659c700 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1ff1e70 0x5597b1ff4260 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7fc374ddf733f5 server_cookie=65f8330b7bdf54aa in_seq=0 out_seq=0 2023-10-08T09:56:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.153+0000 7fe7daff5700 1 -- 172.21.15.97:0/1790742689 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597b1ed3ad0 con 0x5597b1ff1e70 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.153+0000 7fe7daff5700 10 client.15912 handle_command_reply: tid=0 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.153+0000 7fe80925fb80 1 client.15912 shutdown 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.153+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 msgr2=0x5597b1f95d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.153+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8095be0 tx=0x7fe7e80b9040 comp rx=0 tx=0).stop 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 shutdown_connections 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5597b1f892b0 0x5597b1fefd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597b1f87e40 0x5597b1f95d00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5597b1ff1e70 0x5597b1ff4260 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 --2- 172.21.15.97:0/1790742689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597b1f887a0 0x5597b1f96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe7daff5700 0 client.15912 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 >> 172.21.15.97:0/1790742689 conn(0x5597b1efc200 msgr2=0x5597b1f90d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 shutdown_connections 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 1 -- 172.21.15.97:0/1790742689 wait complete. 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe80925fb80 20 client.15912 trim_cache size 0 max 16384 2023-10-08T09:56:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:56:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:56:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:56:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 msgr2=0x7fe7e00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 0x7fe7e00a3940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec011fd0 tx=0x7fe7ec013040 comp rx=0 tx=0).stop 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 msgr2=0x7fe7f8194420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.154+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800da70 tx=0x7fe7e800dd80 comp rx=0 tx=0).stop 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 shutdown_connections 2023-10-08T09:56:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7e00a1480 0x7fe7e00a3940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7f81011e0 0x7fe7f819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f80ff650 0x7fe7f8194420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 --2- 172.21.15.97:0/4075665593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f8100840 0x7fe7f8194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 >> 172.21.15.97:0/4075665593 conn(0x7fe7f80fabe0 msgr2=0x7fe7f8103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 shutdown_connections 2023-10-08T09:56:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:56:50.155+0000 7fe7fd076700 1 -- 172.21.15.97:0/4075665593 wait complete. 2023-10-08T09:56:51.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:51 smithi186 ceph-mon[104393]: pgmap v351: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:51.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:51 smithi097 ceph-mon[99491]: pgmap v351: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:51.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:51 smithi178 ceph-mon[103156]: pgmap v351: 129 pgs: 129 active+clean; 591 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-10-08T09:56:53.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:53 smithi186 ceph-mon[104393]: pgmap v352: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2023-10-08T09:56:53.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:53 smithi097 ceph-mon[99491]: pgmap v352: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2023-10-08T09:56:53.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:53 smithi178 ceph-mon[103156]: pgmap v352: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr:Note: switching to '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'. 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr:state without impacting any branches by switching back to a branch. 2023-10-08T09:56:54.744 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: git switch -c 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr:Or undo this operation with: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: git switch - 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-08T09:56:54.745 INFO:tasks.workunit.client.0.smithi097.stderr: 2023-10-08T09:56:54.746 INFO:tasks.workunit.client.0.smithi097.stderr:HEAD is now at 7b3143bae3b Merge PR #48130 into wip-batrick-testing-20231007.154158 2023-10-08T09:56:54.749 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-08T09:56:54.783 INFO:tasks.workunit.client.0.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-08T09:56:54.785 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-08T09:56:54.785 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-08T09:56:55.313 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-08T09:56:55.363 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-08T09:56:55.400 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-08T09:56:55.401 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-08T09:56:55.402 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-08T09:56:55.438 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-08T09:56:55.441 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T09:56:55.441 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-08T09:56:55.499 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2023-10-08T09:56:55.502 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2023-10-08T09:56:55.502 DEBUG:teuthology.orchestra.run.smithi097:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-10-08T09:56:55.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:55 smithi186 ceph-mon[104393]: pgmap v353: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:55.563 INFO:tasks.workunit.client.0.smithi097.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-10-08T09:56:55.563 INFO:tasks.workunit.client.0.smithi097.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2023-10-08T09:56:55.564 INFO:tasks.workunit.client.0.smithi097.stderr:+ trap -- retry EXIT 2023-10-08T09:56:55.564 INFO:tasks.workunit.client.0.smithi097.stderr:+ rm -rf ffsb 2023-10-08T09:56:55.564 INFO:tasks.workunit.client.0.smithi097.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2023-10-08T09:56:55.567 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into 'ffsb'... 2023-10-08T09:56:55.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:55 smithi097 ceph-mon[99491]: pgmap v353: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:55 smithi178 ceph-mon[103156]: pgmap v353: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-08T09:56:56.527 INFO:tasks.workunit.client.0.smithi097.stderr:+ trap - EXIT 2023-10-08T09:56:56.527 INFO:tasks.workunit.client.0.smithi097.stderr:+ cd ffsb 2023-10-08T09:56:56.527 INFO:tasks.workunit.client.0.smithi097.stderr:+ ./configure 2023-10-08T09:56:56.645 INFO:tasks.workunit.client.0.smithi097.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-08T09:56:57.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:57 smithi186 ceph-mon[104393]: pgmap v354: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:57.653 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether build environment is sane... yes 2023-10-08T09:56:57.662 INFO:tasks.workunit.client.0.smithi097.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-10-08T09:56:57.662 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gawk... gawk 2023-10-08T09:56:57.687 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether make sets $(MAKE)... yes 2023-10-08T09:56:57.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:57 smithi097 ceph-mon[99491]: pgmap v354: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:57.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:57 smithi178 ceph-mon[103156]: pgmap v354: 129 pgs: 129 active+clean; 592 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-08T09:56:57.722 INFO:tasks.workunit.client.0.smithi097.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-08T09:56:57.722 INFO:tasks.workunit.client.0.smithi097.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-08T09:56:57.735 INFO:tasks.workunit.client.0.smithi097.stdout:checking for style of include used by make... GNU 2023-10-08T09:56:57.738 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc... gcc 2023-10-08T09:56:57.793 INFO:tasks.workunit.client.0.smithi097.stdout:checking for C compiler default output file name... a.out 2023-10-08T09:56:57.795 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether the C compiler works... yes 2023-10-08T09:56:57.797 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are cross compiling... no 2023-10-08T09:56:57.833 INFO:tasks.workunit.client.0.smithi097.stdout:checking for suffix of executables... 2023-10-08T09:56:57.867 INFO:tasks.workunit.client.0.smithi097.stdout:checking for suffix of object files... o 2023-10-08T09:56:57.896 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are using the GNU C compiler... yes 2023-10-08T09:56:57.927 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether gcc accepts -g... yes 2023-10-08T09:56:57.968 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc option to accept ISO C89... none needed 2023-10-08T09:56:58.074 INFO:tasks.workunit.client.0.smithi097.stdout:checking dependency style of gcc... gcc3 2023-10-08T09:56:58.121 INFO:tasks.workunit.client.0.smithi097.stdout:checking how to run the C preprocessor... gcc -E 2023-10-08T09:56:58.155 INFO:tasks.workunit.client.0.smithi097.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-10-08T09:56:58.156 INFO:tasks.workunit.client.0.smithi097.stdout:checking for egrep... /usr/bin/grep -E 2023-10-08T09:56:58.164 INFO:tasks.workunit.client.0.smithi097.stdout:checking for AIX... no 2023-10-08T09:56:58.186 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc... (cached) gcc 2023-10-08T09:56:58.194 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-10-08T09:56:58.195 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether gcc accepts -g... (cached) yes 2023-10-08T09:56:58.195 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-10-08T09:56:58.196 INFO:tasks.workunit.client.0.smithi097.stdout:checking dependency style of gcc... (cached) gcc3 2023-10-08T09:56:58.246 INFO:tasks.workunit.client.0.smithi097.stdout:checking for main in -lm... yes 2023-10-08T09:56:58.291 INFO:tasks.workunit.client.0.smithi097.stdout:checking for main in -lpthread... yes 2023-10-08T09:56:58.459 INFO:tasks.workunit.client.0.smithi097.stdout:checking for ANSI C header files... yes 2023-10-08T09:56:58.504 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-10-08T09:56:58.549 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/types.h... yes 2023-10-08T09:56:58.593 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/stat.h... yes 2023-10-08T09:56:58.635 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdlib.h... yes 2023-10-08T09:56:58.678 INFO:tasks.workunit.client.0.smithi097.stdout:checking for string.h... yes 2023-10-08T09:56:58.723 INFO:tasks.workunit.client.0.smithi097.stdout:checking for memory.h... yes 2023-10-08T09:56:58.767 INFO:tasks.workunit.client.0.smithi097.stdout:checking for strings.h... yes 2023-10-08T09:56:58.815 INFO:tasks.workunit.client.0.smithi097.stdout:checking for inttypes.h... yes 2023-10-08T09:56:58.861 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdint.h... yes 2023-10-08T09:56:58.908 INFO:tasks.workunit.client.0.smithi097.stdout:checking for unistd.h... yes 2023-10-08T09:56:58.961 INFO:tasks.workunit.client.0.smithi097.stdout:checking pthread.h usability... yes 2023-10-08T09:56:58.979 INFO:tasks.workunit.client.0.smithi097.stdout:checking pthread.h presence... yes 2023-10-08T09:56:58.980 INFO:tasks.workunit.client.0.smithi097.stdout:checking for pthread.h... yes 2023-10-08T09:56:59.028 INFO:tasks.workunit.client.0.smithi097.stdout:checking fcntl.h usability... yes 2023-10-08T09:56:59.045 INFO:tasks.workunit.client.0.smithi097.stdout:checking fcntl.h presence... yes 2023-10-08T09:56:59.045 INFO:tasks.workunit.client.0.smithi097.stdout:checking for fcntl.h... yes 2023-10-08T09:56:59.093 INFO:tasks.workunit.client.0.smithi097.stdout:checking limits.h usability... yes 2023-10-08T09:56:59.110 INFO:tasks.workunit.client.0.smithi097.stdout:checking limits.h presence... yes 2023-10-08T09:56:59.110 INFO:tasks.workunit.client.0.smithi097.stdout:checking for limits.h... yes 2023-10-08T09:56:59.127 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdint.h... (cached) yes 2023-10-08T09:56:59.175 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/time.h usability... yes 2023-10-08T09:56:59.194 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/time.h presence... yes 2023-10-08T09:56:59.194 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/time.h... yes 2023-10-08T09:56:59.211 INFO:tasks.workunit.client.0.smithi097.stdout:checking for unistd.h... (cached) yes 2023-10-08T09:56:59.252 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/vfs.h usability... yes 2023-10-08T09:56:59.270 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/vfs.h presence... yes 2023-10-08T09:56:59.270 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/vfs.h... yes 2023-10-08T09:56:59.316 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/limits.h usability... no 2023-10-08T09:56:59.333 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/limits.h presence... no 2023-10-08T09:56:59.333 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/limits.h... no 2023-10-08T09:56:59.364 INFO:tasks.workunit.client.0.smithi097.stdout:checking for an ANSI C-conforming const... yes 2023-10-08T09:56:59.395 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-08T09:56:59.396 INFO:tasks.workunit.client.0.smithi097.stdout:checking for function prototypes... yes 2023-10-08T09:56:59.420 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether setvbuf arguments are reversed... no 2023-10-08T09:56:59.486 INFO:tasks.workunit.client.0.smithi097.stdout:checking for system... yes 2023-10-08T09:56:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:56:59 smithi186 ceph-mon[104393]: pgmap v355: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-10-08T09:56:59.545 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gettimeofday... yes 2023-10-08T09:56:59.604 INFO:tasks.workunit.client.0.smithi097.stdout:checking for mkdir... yes 2023-10-08T09:56:59.664 INFO:tasks.workunit.client.0.smithi097.stdout:checking for strerror... yes 2023-10-08T09:56:59.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:56:59 smithi097 ceph-mon[99491]: pgmap v355: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-10-08T09:56:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:56:59 smithi178 ceph-mon[103156]: pgmap v355: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-10-08T09:56:59.723 INFO:tasks.workunit.client.0.smithi097.stdout:checking for open64... yes 2023-10-08T09:56:59.781 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stat64... yes 2023-10-08T09:56:59.841 INFO:tasks.workunit.client.0.smithi097.stdout:checking for fseeko64... yes 2023-10-08T09:56:59.899 INFO:tasks.workunit.client.0.smithi097.stdout:checking for lrand48_r... yes 2023-10-08T09:56:59.959 INFO:tasks.workunit.client.0.smithi097.stdout:checking for srand48_r... yes 2023-10-08T09:56:59.984 INFO:tasks.workunit.client.0.smithi097.stdout:configure: creating ./config.status 2023-10-08T09:57:00.079 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: creating Makefile 2023-10-08T09:57:00.091 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: creating config.h 2023-10-08T09:57:00.106 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: executing depfiles commands 2023-10-08T09:57:00.151 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: executing default commands 2023-10-08T09:57:00.194 INFO:tasks.workunit.client.0.smithi097.stderr:+ make 2023-10-08T09:57:00.200 INFO:tasks.workunit.client.0.smithi097.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2023-10-08T09:57:00.202 INFO:tasks.workunit.client.0.smithi097.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2023-10-08T09:57:00.203 INFO:tasks.workunit.client.0.smithi097.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2023-10-08T09:57:00.204 INFO:tasks.workunit.client.0.smithi097.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2023-10-08T09:57:00.204 INFO:tasks.workunit.client.0.smithi097.stderr: to install the `Automake' and `Perl' packages. Grab them from 2023-10-08T09:57:00.204 INFO:tasks.workunit.client.0.smithi097.stderr: any GNU archive site. 2023-10-08T09:57:00.205 INFO:tasks.workunit.client.0.smithi097.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2023-10-08T09:57:00.206 INFO:tasks.workunit.client.0.smithi097.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2023-10-08T09:57:00.207 INFO:tasks.workunit.client.0.smithi097.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2023-10-08T09:57:00.207 INFO:tasks.workunit.client.0.smithi097.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2023-10-08T09:57:00.208 INFO:tasks.workunit.client.0.smithi097.stderr: You might want to install the `Automake' and `Perl' packages. 2023-10-08T09:57:00.208 INFO:tasks.workunit.client.0.smithi097.stderr: Grab them from any GNU archive site. 2023-10-08T09:57:00.268 INFO:tasks.workunit.client.0.smithi097.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2023-10-08T09:57:00.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:57:00.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 -- 172.21.15.97:0/2200932211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 msgr2=0x7f0da41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 --2- 172.21.15.97:0/2200932211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da41003f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009990 tx=0x7f0d90009ca0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 -- 172.21.15.97:0/2200932211 shutdown_connections 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 --2- 172.21.15.97:0/2200932211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da4107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 --2- 172.21.15.97:0/2200932211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 --2- 172.21.15.97:0/2200932211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da41003f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 -- 172.21.15.97:0/2200932211 >> 172.21.15.97:0/2200932211 conn(0x7f0da40fa390 msgr2=0x7f0da40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 -- 172.21.15.97:0/2200932211 shutdown_connections 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.501+0000 7f0daad05700 1 -- 172.21.15.97:0/2200932211 wait complete. 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 Processor -- start 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 -- start start 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da4196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419e260 con 0x7f0da40fff70 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419e3a0 con 0x7f0da4100930 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0daad05700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419e530 con 0x7f0da40fee20 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57690/0 (socket says 172.21.15.97:57690) 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 -- 172.21.15.97:0/3959116636 learned_addr learned my addr 172.21.15.97:0/3959116636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da4196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da92a2700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 -- 172.21.15.97:0/3959116636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 msgr2=0x7f0da4196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da4196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 -- 172.21.15.97:0/3959116636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 msgr2=0x7f0da419dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da419dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 -- 172.21.15.97:0/3959116636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900095f0 con 0x7f0da40fff70 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da92a2700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da419dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:00.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da4196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.502+0000 7f0da3fff700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c010 tx=0x7f0d90005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.503+0000 7f0da1ffb700 1 -- 172.21.15.97:0/3959116636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001d070 con 0x7f0da40fff70 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.503+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da419e7b0 con 0x7f0da40fff70 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.503+0000 7f0da1ffb700 1 -- 172.21.15.97:0/3959116636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d90004ca0 con 0x7f0da40fff70 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.503+0000 7f0da1ffb700 1 -- 172.21.15.97:0/3959116636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90005610 con 0x7f0da40fff70 2023-10-08T09:57:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.503+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da419ebd0 con 0x7f0da40fff70 2023-10-08T09:57:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.505+0000 7f0da1ffb700 1 -- 172.21.15.97:0/3959116636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0d9000fc70 con 0x7f0da40fff70 2023-10-08T09:57:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.505+0000 7f0da1ffb700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 0x7f0d940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.505+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 0x7f0d940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.505+0000 7f0da1ffb700 1 -- 172.21.15.97:0/3959116636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d900ce340 con 0x7f0da40fff70 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.506+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 0x7f0d940a3940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d98005950 tx=0x7f0d98005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.506+0000 7f0db6eeeb80 1 Processor -- start 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.506+0000 7f0db6eeeb80 1 -- start start 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a18362060 0x563a18362480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18363c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a182ca460 con 0x563a183629c0 2023-10-08T09:57:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a182bbc20 con 0x563a18362060 2023-10-08T09:57:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a182b9540 con 0x563a18363800 2023-10-08T09:57:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57698/0 (socket says 172.21.15.97:57698) 2023-10-08T09:57:00.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 -- 172.21.15.97:0/2050080932 learned_addr learned my addr 172.21.15.97:0/2050080932 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a18362060 0x563a18362480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da92a2700 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18363c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 -- 172.21.15.97:0/2050080932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 msgr2=0x563a18363c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18363c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 -- 172.21.15.97:0/2050080932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a18362060 msgr2=0x563a18362480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a18362060 0x563a18362480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 -- 172.21.15.97:0/2050080932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a180dfde0 con 0x563a183629c0 2023-10-08T09:57:00.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0da3fff700 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d9009b0c0 tx=0x7f0d90009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d687abfe703e5f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0d8effd700 1 -- 172.21.15.97:0/2050080932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900ce160 con 0x563a183629c0 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0d8effd700 1 -- 172.21.15.97:0/2050080932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9009b3c0 con 0x563a183629c0 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.507+0000 7f0d8effd700 1 -- 172.21.15.97:0/2050080932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9009f160 con 0x563a183629c0 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/2050080932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 msgr2=0x563a18363270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d9009b0c0 tx=0x7f0d90009ca0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/2050080932 shutdown_connections 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18363c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a18362060 0x563a18362480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/2050080932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a183629c0 0x563a18363270 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/2050080932 >> 172.21.15.97:0/2050080932 conn(0x563a182d6260 msgr2=0x563a1835ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/2050080932 shutdown_connections 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/2050080932 wait complete. 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 -- start start 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.508+0000 7f0db6eeeb80 1 Processor -- start 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a1836bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18372f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a182b4e30 con 0x563a18362060 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a18251710 con 0x563a183629c0 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a182513b0 con 0x563a18363800 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0db6eeeb80 10 client.0 _ll_register_callbacks cb 0x563a180ae800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57702/0 (socket says 172.21.15.97:57702) 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 -- 172.21.15.97:0/31575381 learned_addr learned my addr 172.21.15.97:0/31575381 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da92a2700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18372f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da3fff700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a1836bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 -- 172.21.15.97:0/31575381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 msgr2=0x563a18372f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18372f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 -- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 msgr2=0x563a1836bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a1836bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 -- 172.21.15.97:0/31575381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da419ebd0 con 0x563a18362060 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da3fff700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a1836bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0d980109e0 tx=0x7f0d98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1d918b020a07ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.509+0000 7f0d83fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004a30 con 0x563a18362060 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d98004b90 con 0x563a18362060 2023-10-08T09:57:00.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a1827ceb0 con 0x563a18362060 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d98004d50 con 0x563a18362060 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 10 client.15924 fetch_fsmap learned FSMap version 24 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563a180c8c60 con 0x563a18362060 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a1827ceb0 con 0x563a18362060 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d98005490 con 0x563a18362060 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:00.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 20 client.15924 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:00.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a18379cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 4 client.15924 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:00.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a182ade10 con 0x563a183778d0 2023-10-08T09:57:00.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.510+0000 7f0da3fff700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a18379cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.528 INFO:tasks.workunit.client.0.smithi097.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2023-10-08T09:57:00.528 INFO:tasks.workunit.client.0.smithi097.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2023-10-08T09:57:00.528 INFO:tasks.workunit.client.0.smithi097.stderr:If you have problems, you may need to regenerate the build system entirely. 2023-10-08T09:57:00.528 INFO:tasks.workunit.client.0.smithi097.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2023-10-08T09:57:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.511+0000 7f0d83fff700 10 client.15924 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:00.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.511+0000 7f0da3fff700 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a18379cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692d41a20fac92f5 server_cookie=7dbf27b0aa92d565 in_seq=0 out_seq=0 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.511+0000 7f0d83fff700 1 -- 172.21.15.97:0/31575381 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a182ade10 con 0x563a183778d0 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.511+0000 7f0d83fff700 10 client.15924 handle_command_reply: tid=0 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 client.15924 shutdown 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 msgr2=0x563a1836ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0d980109e0 tx=0x7f0d98004750 comp rx=0 tx=0).stop 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 shutdown_connections 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a18379cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a18372f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0d83fff700 0 client.15924 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:00.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a1836ba10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/31575381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a1836bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 >> 172.21.15.97:0/31575381 conn(0x563a182d6260 msgr2=0x563a1835c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 shutdown_connections 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/31575381 wait complete. 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.512+0000 7f0db6eeeb80 20 client.15924 trim_cache size 0 max 16384 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 Processor -- start 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 -- start start 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a1835cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a18135920 con 0x563a18362060 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a18362e90 con 0x563a183629c0 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1835e610 con 0x563a18363800 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57710/0 (socket says 172.21.15.97:57710) 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 -- 172.21.15.97:0/3055279301 learned_addr learned my addr 172.21.15.97:0/3055279301 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da92a2700 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a1835cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 -- 172.21.15.97:0/3055279301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 msgr2=0x563a1835cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a1835cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da3fff700 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 -- 172.21.15.97:0/3055279301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 msgr2=0x563a18372f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 -- 172.21.15.97:0/3055279301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a18106250 con 0x563a18362060 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.529+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0d98013040 tx=0x7f0d9800a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a5be047e28a463 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0d8f7fe700 1 -- 172.21.15.97:0/3055279301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9801e980 con 0x563a18362060 2023-10-08T09:57:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0d8f7fe700 1 -- 172.21.15.97:0/3055279301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9801d6a0 con 0x563a18362060 2023-10-08T09:57:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0d8f7fe700 1 -- 172.21.15.97:0/3055279301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98034690 con 0x563a18362060 2023-10-08T09:57:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/3055279301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 msgr2=0x563a18379cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0d98013040 tx=0x7f0d9800a540 comp rx=0 tx=0).stop 2023-10-08T09:57:00.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/3055279301 shutdown_connections 2023-10-08T09:57:00.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18363800 0x563a1835cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18362060 0x563a18379cf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/3055279301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/3055279301 >> 172.21.15.97:0/3055279301 conn(0x563a182d6260 msgr2=0x563a1837e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/3055279301 shutdown_connections 2023-10-08T09:57:00.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.530+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/3055279301 wait complete. 2023-10-08T09:57:00.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 -- start start 2023-10-08T09:57:00.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 Processor -- start 2023-10-08T09:57:00.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18362060 0x563a183726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1836f810 con 0x563a18363800 2023-10-08T09:57:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1836f980 con 0x563a183629c0 2023-10-08T09:57:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a1836fb40 con 0x563a18362060 2023-10-08T09:57:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0db6eeeb80 10 client.0 _ll_register_callbacks cb 0x563a180ae800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0da92a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0da92a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57720/0 (socket says 172.21.15.97:57720) 2023-10-08T09:57:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0da92a2700 1 -- 172.21.15.97:0/631018909 learned_addr learned my addr 172.21.15.97:0/631018909 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0da3fff700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.531+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18362060 0x563a183726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 -- 172.21.15.97:0/631018909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18362060 msgr2=0x563a183726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18362060 0x563a183726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 -- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 msgr2=0x563a18372c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:00.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 -- 172.21.15.97:0/631018909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a180c8c60 con 0x563a18363800 2023-10-08T09:57:00.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0da92a2700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c007bf0 tx=0x7f0d9c007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08c98e23ae33dbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:00.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00e040 con 0x563a18363800 2023-10-08T09:57:00.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d9c004db0 con 0x563a18363800 2023-10-08T09:57:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d9009b3c0 con 0x563a18363800 2023-10-08T09:57:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d9c003a00 con 0x563a18363800 2023-10-08T09:57:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 10 client.15930 fetch_fsmap learned FSMap version 24 2023-10-08T09:57:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563a180de2b0 con 0x563a18363800 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9009b3c0 con 0x563a18363800 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d9c0175d0 con 0x563a18363800 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 20 client.15930 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a183ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 4 client.15930 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:57:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.532+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a182adb30 con 0x563a183778d0 2023-10-08T09:57:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a183ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0d83fff700 10 client.15930 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0da8aa1700 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a183ced30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e1258351478f1f server_cookie=d6099000c3778f54 in_seq=0 out_seq=0 2023-10-08T09:57:00.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0d83fff700 1 -- 172.21.15.97:0/631018909 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563a182adb30 con 0x563a183778d0 2023-10-08T09:57:00.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0d83fff700 10 client.15930 handle_command_reply: tid=0 2023-10-08T09:57:00.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 client.15930 shutdown 2023-10-08T09:57:00.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 msgr2=0x563a1836f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c007bf0 tx=0x7f0d9c007c20 comp rx=0 tx=0).stop 2023-10-08T09:57:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 shutdown_connections 2023-10-08T09:57:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563a18363800 0x563a1836f2d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x563a183778d0 0x563a183ced30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563a18362060 0x563a183726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0d83fff700 0 client.15930 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 --2- 172.21.15.97:0/631018909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a183629c0 0x563a18372c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 >> 172.21.15.97:0/631018909 conn(0x563a182d6260 msgr2=0x563a183666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 shutdown_connections 2023-10-08T09:57:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.533+0000 7f0db6eeeb80 1 -- 172.21.15.97:0/631018909 wait complete. 2023-10-08T09:57:00.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0db6eeeb80 20 client.15930 trim_cache size 0 max 16384 2023-10-08T09:57:00.556 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:57:00.557 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2023-10-08T09:57:00.557 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2023-10-08T09:57:00.557 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 msgr2=0x7f0d940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 0x7f0d940a3940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d98005950 tx=0x7f0d98005f50 comp rx=0 tx=0).stop 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 msgr2=0x7f0da4196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c010 tx=0x7f0d90005e80 comp rx=0 tx=0).stop 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 shutdown_connections 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0d940a1480 0x7f0d940a3940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0da4100930 0x7f0da419dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da40fee20 0x7f0da4196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 --2- 172.21.15.97:0/3959116636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40fff70 0x7f0da4196c70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 >> 172.21.15.97:0/3959116636 conn(0x7f0da40fa390 msgr2=0x7f0da40fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 shutdown_connections 2023-10-08T09:57:00.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.534+0000 7f0daad05700 1 -- 172.21.15.97:0/3959116636 wait complete. 2023-10-08T09:57:00.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T09:57:00.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:57:00.646 INFO:tasks.workunit.client.0.smithi097.stdout:/bin/sh ./config.status --recheck 2023-10-08T09:57:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- 172.21.15.97:0/1513095317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 msgr2=0x7f7e9c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1513095317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c0fff10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009990 tx=0x7f7e8c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- 172.21.15.97:0/1513095317 shutdown_connections 2023-10-08T09:57:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1513095317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c1011c0 0x7f7e9c107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1513095317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c0fff10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1513095317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c100800 0x7f7e9c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- 172.21.15.97:0/1513095317 >> 172.21.15.97:0/1513095317 conn(0x7f7e9c0fa2c0 msgr2=0x7f7e9c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- 172.21.15.97:0/1513095317 shutdown_connections 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- 172.21.15.97:0/1513095317 wait complete. 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 Processor -- start 2023-10-08T09:57:00.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.684+0000 7f7ea2585700 1 -- start start 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c100800 0x7f7e9c195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c1011c0 0x7f7e9c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19d080 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19d1c0 con 0x7f7e9c1011c0 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea2585700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19d350 con 0x7f7e9c100800 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57730/0 (socket says 172.21.15.97:57730) 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 -- 172.21.15.97:0/1627740251 learned_addr learned my addr 172.21.15.97:0/1627740251 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7ea0b22700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c1011c0 0x7f7e9c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9b7fe700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c100800 0x7f7e9c195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 -- 172.21.15.97:0/1627740251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c100800 msgr2=0x7f7e9c195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c100800 0x7f7e9c195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 -- 172.21.15.97:0/1627740251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c1011c0 msgr2=0x7f7e9c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c1011c0 0x7f7e9c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 -- 172.21.15.97:0/1627740251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0095f0 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.685+0000 7f7e9bfff700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000c00 tx=0x7f7e8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.686+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01d070 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.686+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c19d5d0 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.686+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c005030 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.686+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c004bc0 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.686+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c19d990 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.687+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c106a60 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.688+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7e8c004d20 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.688+0000 7f7e997fa700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 0x7f7e840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.688+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7e8c0ce420 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.688+0000 7f7e9b7fe700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 0x7f7e840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.689+0000 7f7e9b7fe700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 0x7f7e840a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0ff810 tx=0x7f7e90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.692+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e8c0977e0 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.694 INFO:tasks.workunit.client.0.smithi097.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2023-10-08T09:57:00.789 INFO:tasks.workunit.client.0.smithi097.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-08T09:57:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.938+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c10ae10 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.939+0000 7f7e997fa700 1 -- 172.21.15.97:0/1627740251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11334 (secure 0 0 0) 0x7f7e8c0051a0 con 0x7f7e9c0ffaf0 2023-10-08T09:57:00.944 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:57:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:54:24.185261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:57:00.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.949+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 msgr2=0x7f7e840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.949+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 0x7f7e840a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0ff810 tx=0x7f7e90009260 comp rx=0 tx=0).stop 2023-10-08T09:57:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.949+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 msgr2=0x7f7e9c195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.949+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000c00 tx=0x7f7e8c004730 comp rx=0 tx=0).stop 2023-10-08T09:57:00.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 shutdown_connections 2023-10-08T09:57:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7e840a1480 0x7f7e840a3940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c1011c0 0x7f7e9c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e9c0ffaf0 0x7f7e9c195550 secure :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000c00 tx=0x7f7e8c004730 comp rx=0 tx=0).stop 2023-10-08T09:57:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 --2- 172.21.15.97:0/1627740251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e9c100800 0x7f7e9c195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.952+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 >> 172.21.15.97:0/1627740251 conn(0x7f7e9c0fa2c0 msgr2=0x7f7e9c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:00.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.956+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 shutdown_connections 2023-10-08T09:57:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:00.961+0000 7f7ea2585700 1 -- 172.21.15.97:0/1627740251 wait complete. 2023-10-08T09:57:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2023-10-08T09:57:00.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T09:57:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 -- 172.21.15.97:0/262153258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7954100890 msgr2=0x7f7954100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 --2- 172.21.15.97:0/262153258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7954100890 0x7f7954100cf0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f79480099c0 tx=0x7f7948009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 -- 172.21.15.97:0/262153258 shutdown_connections 2023-10-08T09:57:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 --2- 172.21.15.97:0/262153258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954101230 0x7f7954107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 --2- 172.21.15.97:0/262153258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79540ff6a0 0x7f79540ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 --2- 172.21.15.97:0/262153258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7954100890 0x7f7954100cf0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 -- 172.21.15.97:0/262153258 >> 172.21.15.97:0/262153258 conn(0x7f79540fac10 msgr2=0x7f79540fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 -- 172.21.15.97:0/262153258 shutdown_connections 2023-10-08T09:57:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.128+0000 7f7958fa1700 1 -- 172.21.15.97:0/262153258 wait complete. 2023-10-08T09:57:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.129+0000 7f7958fa1700 1 Processor -- start 2023-10-08T09:57:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.129+0000 7f7958fa1700 1 -- start start 2023-10-08T09:57:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954100890 0x7f7954195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954101230 0x7f795419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419d150 con 0x7f79540ff6a0 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419d290 con 0x7f7954100890 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419d420 con 0x7f7954101230 2023-10-08T09:57:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57766/0 (socket says 172.21.15.97:57766) 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 -- 172.21.15.97:0/674071483 learned_addr learned my addr 172.21.15.97:0/674071483 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7951d9b700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954100890 0x7f7954195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7952d9d700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954101230 0x7f795419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 -- 172.21.15.97:0/674071483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954101230 msgr2=0x7f795419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954101230 0x7f795419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 -- 172.21.15.97:0/674071483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954100890 msgr2=0x7f7954195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954100890 0x7f7954195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 -- 172.21.15.97:0/674071483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480095f0 con 0x7f79540ff6a0 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f795259c700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7944007b50 tx=0x7f7944007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f79437fe700 1 -- 172.21.15.97:0/674071483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944011070 con 0x7f79540ff6a0 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f79437fe700 1 -- 172.21.15.97:0/674071483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794400c2e0 con 0x7f79540ff6a0 2023-10-08T09:57:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7954067890 con 0x7f79540ff6a0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.130+0000 7f79437fe700 1 -- 172.21.15.97:0/674071483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794400e580 con 0x7f79540ff6a0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.131+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7954067da0 con 0x7f79540ff6a0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.133+0000 7f79437fe700 1 -- 172.21.15.97:0/674071483 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7944003f90 con 0x7f79540ff6a0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.133+0000 7f79437fe700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 0x7f793c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f79437fe700 1 -- 172.21.15.97:0/674071483 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f79440ce2d0 con 0x7f79540ff6a0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f7951d9b700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 0x7f793c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 Processor -- start 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f7951d9b700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 0x7f793c0a3940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7948009990 tx=0x7f7948000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 -- start start 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb8315460 con 0x558bb83ad9c0 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb8306c20 con 0x558bb83ad060 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.134+0000 7f796518ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb8304540 con 0x558bb83ae800 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57776/0 (socket says 172.21.15.97:57776) 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f795259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 -- 172.21.15.97:0/1252804382 learned_addr learned my addr 172.21.15.97:0/1252804382 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7952d9d700 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 -- 172.21.15.97:0/1252804382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 msgr2=0x558bb83aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 -- 172.21.15.97:0/1252804382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 msgr2=0x558bb83ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 -- 172.21.15.97:0/1252804382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb812ade0 con 0x558bb83ad9c0 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f795259c700 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7951d9b700 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7948015000 tx=0x7f794801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c044ae13b8f1a1d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7940ff9700 1 -- 172.21.15.97:0/1252804382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948029070 con 0x558bb83ad9c0 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7940ff9700 1 -- 172.21.15.97:0/1252804382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794801e040 con 0x558bb83ad9c0 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.135+0000 7f7940ff9700 1 -- 172.21.15.97:0/1252804382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794800f920 con 0x558bb83ad9c0 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- 172.21.15.97:0/1252804382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 msgr2=0x558bb83ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7948015000 tx=0x7f794801f000 comp rx=0 tx=0).stop 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- 172.21.15.97:0/1252804382 shutdown_connections 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 --2- 172.21.15.97:0/1252804382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83ae270 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- 172.21.15.97:0/1252804382 >> 172.21.15.97:0/1252804382 conn(0x558bb8321260 msgr2=0x558bb83a7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- 172.21.15.97:0/1252804382 shutdown_connections 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- 172.21.15.97:0/1252804382 wait complete. 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 -- start start 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.136+0000 7f796518ab80 1 Processor -- start 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83b7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb82ffe30 con 0x558bb83ad060 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb829c710 con 0x558bb83ad9c0 2023-10-08T09:57:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb829c3b0 con 0x558bb83ae800 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 10 client.0 _ll_register_callbacks cb 0x558bb80f9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57780/0 (socket says 172.21.15.97:57780) 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 -- 172.21.15.97:0/2474773608 learned_addr learned my addr 172.21.15.97:0/2474773608 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7951d9b700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7952d9d700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 -- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 msgr2=0x558bb83be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 -- 172.21.15.97:0/2474773608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 msgr2=0x558bb83b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 -- 172.21.15.97:0/2474773608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7954067da0 con 0x558bb83ad060 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7952d9d700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f795259c700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7944007b20 tx=0x7f794400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315e5d000ede97f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7939ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79440ce170 con 0x558bb83ad060 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f794409f540 con 0x558bb83ad060 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bb82c7eb0 con 0x558bb83ad060 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79440a36f0 con 0x558bb83ad060 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 10 client.15948 fetch_fsmap learned FSMap version 24 2023-10-08T09:57:01.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.137+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f7954108430 con 0x558bb83ad060 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558bb82c7eb0 con 0x558bb83ad060 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f794409b650 con 0x558bb83ad060 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 10 client.15948 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 20 client.15948 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb83c2a20 0x558bb83c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 4 client.15948 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bb82f8e10 con 0x558bb83c2a20 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.138+0000 7f7951d9b700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb83c2a20 0x558bb83c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f7939ffb700 10 client.15948 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f7951d9b700 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb83c2a20 0x558bb83c4e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c648f7a927d66dbb server_cookie=8681e3fb5938721 in_seq=0 out_seq=0 2023-10-08T09:57:01.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f7939ffb700 1 -- 172.21.15.97:0/2474773608 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558bb82f8e10 con 0x558bb83c2a20 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f7939ffb700 10 client.15948 handle_command_reply: tid=0 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f796518ab80 1 client.15948 shutdown 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 msgr2=0x558bb83b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.139+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7944007b20 tx=0x7f794400cf80 comp rx=0 tx=0).stop 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 shutdown_connections 2023-10-08T09:57:01.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb83c2a20 0x558bb83c4e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad060 0x558bb83b6cf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 --2- 172.21.15.97:0/2474773608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f7939ffb700 0 client.15948 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 >> 172.21.15.97:0/2474773608 conn(0x558bb8321260 msgr2=0x558bb83a74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 shutdown_connections 2023-10-08T09:57:01.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 1 -- 172.21.15.97:0/2474773608 wait complete. 2023-10-08T09:57:01.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.140+0000 7f796518ab80 20 client.15948 trim_cache size 0 max 16384 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 Processor -- start 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 -- start start 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ad060 0x558bb83b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb8180920 con 0x558bb83ae800 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb83ae060 con 0x558bb83ad9c0 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f796518ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb83b4970 con 0x558bb83ad060 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57792/0 (socket says 172.21.15.97:57792) 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 -- 172.21.15.97:0/4126162667 learned_addr learned my addr 172.21.15.97:0/4126162667 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7951d9b700 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f795259c700 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ad060 0x558bb83b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 -- 172.21.15.97:0/4126162667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ad060 msgr2=0x558bb83b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ad060 0x558bb83b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 -- 172.21.15.97:0/4126162667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 msgr2=0x558bb83be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.153+0000 7f7952d9d700 1 -- 172.21.15.97:0/4126162667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb8113c60 con 0x558bb83ae800 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f7952d9d700 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f794c0078e0 tx=0x7f794c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20ba48e0c959294 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f79417fa700 1 -- 172.21.15.97:0/4126162667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00e040 con 0x558bb83ae800 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f79417fa700 1 -- 172.21.15.97:0/4126162667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794c01b070 con 0x558bb83ae800 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f79417fa700 1 -- 172.21.15.97:0/4126162667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c022e20 con 0x558bb83ae800 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 -- 172.21.15.97:0/4126162667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 msgr2=0x558bb83b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f794c0078e0 tx=0x7f794c009920 comp rx=0 tx=0).stop 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 -- 172.21.15.97:0/4126162667 shutdown_connections 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ae800 0x558bb83b1700 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ad060 0x558bb83b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 --2- 172.21.15.97:0/4126162667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad9c0 0x558bb83be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 -- 172.21.15.97:0/4126162667 >> 172.21.15.97:0/4126162667 conn(0x558bb8321260 msgr2=0x558bb83c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 -- 172.21.15.97:0/4126162667 shutdown_connections 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.154+0000 7f796518ab80 1 -- 172.21.15.97:0/4126162667 wait complete. 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 -- start start 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 Processor -- start 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb83bb480 con 0x558bb83ad9c0 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb83bb5f0 con 0x558bb83ad060 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb8416840 con 0x558bb83ae800 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f796518ab80 10 client.0 _ll_register_callbacks cb 0x7f794409b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57796/0 (socket says 172.21.15.97:57796) 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f795259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 -- 172.21.15.97:0/249095341 learned_addr learned my addr 172.21.15.97:0/249095341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7952d9d700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 -- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 msgr2=0x558bb83baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 -- 172.21.15.97:0/249095341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 msgr2=0x558bb83bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.155+0000 7f7951d9b700 1 -- 172.21.15.97:0/249095341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c0075c0 con 0x558bb83ad9c0 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7951d9b700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f794802f040 tx=0x7f794800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4c49fcf7edaf10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948029040 con 0x558bb83ad9c0 2023-10-08T09:57:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f794801e040 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f794803b460 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f794803ba80 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 10 client.15954 fetch_fsmap learned FSMap version 24 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558bb8151250 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794801e040 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7948005350 con 0x558bb83ad9c0 2023-10-08T09:57:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 10 client.15954 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T09:57:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 10 client.15954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:57:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 20 client.15954 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb8417d60 0x558bb841a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 4 client.15954 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:57:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558bb82f8b30 con 0x558bb8417d60 2023-10-08T09:57:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.156+0000 7f795259c700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb8417d60 0x558bb841a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f7939ffb700 10 client.15954 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:57:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f795259c700 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb8417d60 0x558bb841a180 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1c0cbe82a16004b server_cookie=3aaecb23e87ae3e6 in_seq=0 out_seq=0 2023-10-08T09:57:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f7939ffb700 1 -- 172.21.15.97:0/249095341 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558bb82f8b30 con 0x558bb8417d60 2023-10-08T09:57:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f7939ffb700 10 client.15954 handle_command_reply: tid=0 2023-10-08T09:57:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 client.15954 shutdown 2023-10-08T09:57:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 msgr2=0x558bb83baa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f794802f040 tx=0x7f794800bd70 comp rx=0 tx=0).stop 2023-10-08T09:57:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 shutdown_connections 2023-10-08T09:57:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x558bb8417d60 0x558bb841a180 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558bb83ae800 0x558bb83baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558bb83ad060 0x558bb83bdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 --2- 172.21.15.97:0/249095341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb83ad9c0 0x558bb83baa00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 >> 172.21.15.97:0/249095341 conn(0x558bb8321260 msgr2=0x558bb83b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f7939ffb700 0 client.15954 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:57:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 shutdown_connections 2023-10-08T09:57:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 1 -- 172.21.15.97:0/249095341 wait complete. 2023-10-08T09:57:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.157+0000 7f796518ab80 20 client.15954 trim_cache size 0 max 16384 2023-10-08T09:57:01.179 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:57:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 msgr2=0x7f793c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 0x7f793c0a3940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7948009990 tx=0x7f7948000bc0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 msgr2=0x7f7954195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7944007b50 tx=0x7f7944007e60 comp rx=0 tx=0).stop 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 shutdown_connections 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f793c0a1480 0x7f793c0a3940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7954101230 0x7f795419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79540ff6a0 0x7f7954195620 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 --2- 172.21.15.97:0/674071483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7954100890 0x7f7954195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 >> 172.21.15.97:0/674071483 conn(0x7f79540fac10 msgr2=0x7f7954103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 shutdown_connections 2023-10-08T09:57:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.158+0000 7f7958fa1700 1 -- 172.21.15.97:0/674071483 wait complete. 2023-10-08T09:57:01.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T09:57:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.312+0000 7f6502366700 1 -- 172.21.15.97:0/1426032351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 msgr2=0x7f64fc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.312+0000 7f6502366700 1 --2- 172.21.15.97:0/1426032351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc100be0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0099c0 tx=0x7f64ec009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.313+0000 7f6502366700 1 -- 172.21.15.97:0/1426032351 shutdown_connections 2023-10-08T09:57:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.313+0000 7f6502366700 1 --2- 172.21.15.97:0/1426032351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64fc102350 0x7f64fc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.313+0000 7f6502366700 1 --2- 172.21.15.97:0/1426032351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc100be0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.313+0000 7f6502366700 1 --2- 172.21.15.97:0/1426032351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.313+0000 7f6502366700 1 -- 172.21.15.97:0/1426032351 >> 172.21.15.97:0/1426032351 conn(0x7f64fc0fbd30 msgr2=0x7f64fc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.315+0000 7f6502366700 1 -- 172.21.15.97:0/1426032351 shutdown_connections 2023-10-08T09:57:01.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.315+0000 7f6502366700 1 -- 172.21.15.97:0/1426032351 wait complete. 2023-10-08T09:57:01.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 Processor -- start 2023-10-08T09:57:01.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 -- start start 2023-10-08T09:57:01.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc194640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64fc102350 0x7f64fc19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc19c060 con 0x7f64fc1007c0 2023-10-08T09:57:01.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc19c1d0 con 0x7f64fc1019b0 2023-10-08T09:57:01.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.316+0000 7f6502366700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc19c390 con 0x7f64fc102350 2023-10-08T09:57:01.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37780/0 (socket says 172.21.15.97:37780) 2023-10-08T09:57:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 -- 172.21.15.97:0/660878849 learned_addr learned my addr 172.21.15.97:0/660878849 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64fbfff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc194640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 -- 172.21.15.97:0/660878849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64fc102350 msgr2=0x7f64fc19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64fc102350 0x7f64fc19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 -- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 msgr2=0x7f64fc194640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc194640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64f3fff700 1 -- 172.21.15.97:0/660878849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec0095f0 con 0x7f64fc1019b0 2023-10-08T09:57:01.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.317+0000 7f64fbfff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:01.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f64f3fff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64e400e9a0 tx=0x7f64e400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f64f9ffb700 1 -- 172.21.15.97:0/660878849 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4009790 con 0x7f64fc1019b0 2023-10-08T09:57:01.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f64f9ffb700 1 -- 172.21.15.97:0/660878849 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e400cbd0 con 0x7f64fc1019b0 2023-10-08T09:57:01.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc1ad690 con 0x7f64fc1019b0 2023-10-08T09:57:01.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f64f9ffb700 1 -- 172.21.15.97:0/660878849 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4018680 con 0x7f64fc1019b0 2023-10-08T09:57:01.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.318+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1adb70 con 0x7f64fc1019b0 2023-10-08T09:57:01.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.322+0000 7f64f9ffb700 1 -- 172.21.15.97:0/660878849 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f64e40187e0 con 0x7f64fc1019b0 2023-10-08T09:57:01.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.322+0000 7f64f9ffb700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 0x7f64e80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f64fbfff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 0x7f64e80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f64f9ffb700 1 -- 172.21.15.97:0/660878849 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f64e4014070 con 0x7f64fc1019b0 2023-10-08T09:57:01.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 Processor -- start 2023-10-08T09:57:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 -- start start 2023-10-08T09:57:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 0x557f0fa30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa30800 0x557f0fa30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f997460 con 0x557f0fa30800 2023-10-08T09:57:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f988c20 con 0x557f0fa2f060 2023-10-08T09:57:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f650e54fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f986540 con 0x557f0fa2f9c0 2023-10-08T09:57:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.323+0000 7f64fbfff700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 0x7f64e80a3a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37786/0 (socket says 172.21.15.97:37786) 2023-10-08T09:57:01.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 -- 172.21.15.97:0/2048135168 learned_addr learned my addr 172.21.15.97:0/2048135168 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 0x557f0fa30270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 -- 172.21.15.97:0/2048135168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 msgr2=0x557f0fa30270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 0x557f0fa30270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 -- 172.21.15.97:0/2048135168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa30800 msgr2=0x557f0fa30c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa30800 0x557f0fa30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 -- 172.21.15.97:0/2048135168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f0f7acde0 con 0x557f0fa2f060 2023-10-08T09:57:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 0x557f0fa30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:57:01.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.324+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64ec015000 tx=0x7f64ec01a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2205a3a8ba580f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.326+0000 7f64f27fc700 1 -- 172.21.15.97:0/2048135168 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec004b30 con 0x557f0fa2f060 2023-10-08T09:57:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.326+0000 7f64f27fc700 1 -- 172.21.15.97:0/2048135168 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64ec01e040 con 0x557f0fa2f060 2023-10-08T09:57:01.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.326+0000 7f64f27fc700 1 -- 172.21.15.97:0/2048135168 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec010470 con 0x557f0fa2f060 2023-10-08T09:57:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- 172.21.15.97:0/2048135168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 msgr2=0x557f0fa2f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64ec015000 tx=0x7f64ec01a070 comp rx=0 tx=0).stop 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- 172.21.15.97:0/2048135168 shutdown_connections 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa30800 0x557f0fa30c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2048135168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f9c0 0x557f0fa30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- 172.21.15.97:0/2048135168 >> 172.21.15.97:0/2048135168 conn(0x557f0f9a3260 msgr2=0x557f0fa29e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- 172.21.15.97:0/2048135168 shutdown_connections 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- 172.21.15.97:0/2048135168 wait complete. 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.327+0000 7f650e54fb80 1 -- start start 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 Processor -- start 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f981e30 con 0x557f0fa2f9c0 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f91e710 con 0x557f0fa2f060 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f91e3b0 con 0x557f0fa30800 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.328+0000 7f650e54fb80 10 client.0 _ll_register_callbacks cb 0x557f0f77b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f6500903700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f6500903700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59522/0 (socket says 172.21.15.97:59522) 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f6500903700 1 -- 172.21.15.97:0/2585484343 learned_addr learned my addr 172.21.15.97:0/2585484343 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 -- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 msgr2=0x557f0fa40320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 -- 172.21.15.97:0/2585484343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 msgr2=0x557f0fa38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 -- 172.21.15.97:0/2585484343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc109550 con 0x557f0fa2f9c0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f6500903700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.329+0000 7f64f3fff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa392e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f64e40ca850 tx=0x7f64e409a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680d7514f6cf4a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f64db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e409e170 con 0x557f0fa2f9c0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f64e40a23b0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f0f949eb0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e400cd40 con 0x557f0fa2f9c0 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f650e54fb80 10 client.15963 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557f0f795c60 con 0x557f0fa2f9c0 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.330+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f0f949eb0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f64e4098f80 con 0x557f0fa2f9c0 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 20 client.15963 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa44c10 0x557f0fa47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 4 client.15963 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f0f97ae10 con 0x557f0fa44c10 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.331+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa44c10 0x557f0fa47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.332+0000 7f64fbfff700 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa44c10 0x557f0fa47030 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9334e4329e4a9b5 server_cookie=d3bbcaa94d00bb2c in_seq=0 out_seq=0 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.332+0000 7f64db7fe700 10 client.15963 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f64db7fe700 1 -- 172.21.15.97:0/2585484343 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f0f97ae10 con 0x557f0fa44c10 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f64db7fe700 10 client.15963 handle_command_reply: tid=0 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 client.15963 shutdown 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 msgr2=0x557f0fa392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa392e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f64e40ca850 tx=0x7f64e409a560 comp rx=0 tx=0).stop 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 shutdown_connections 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa40320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa44c10 0x557f0fa47030 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 --2- 172.21.15.97:0/2585484343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa392e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f64db7fe700 0 client.15963 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:57:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 >> 172.21.15.97:0/2585484343 conn(0x557f0f9a3260 msgr2=0x557f0fa294d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.333+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 shutdown_connections 2023-10-08T09:57:01.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.334+0000 7f650e54fb80 1 -- 172.21.15.97:0/2585484343 wait complete. 2023-10-08T09:57:01.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.334+0000 7f650e54fb80 20 client.15963 trim_cache size 0 max 16384 2023-10-08T09:57:01.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.356+0000 7f650e54fb80 1 Processor -- start 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 -- start start 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa25fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0f802920 con 0x557f0fa2f9c0 2023-10-08T09:57:01.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0fa38910 con 0x557f0fa2f060 2023-10-08T09:57:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f650e54fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0fa24570 con 0x557f0fa30800 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa40240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57834/0 (socket says 172.21.15.97:57834) 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f64f3fff700 1 -- 172.21.15.97:0/1002178185 learned_addr learned my addr 172.21.15.97:0/1002178185 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f6500903700 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa25fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.357+0000 7f64fbfff700 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 -- 172.21.15.97:0/1002178185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 msgr2=0x557f0fa2f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 -- 172.21.15.97:0/1002178185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 msgr2=0x557f0fa40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa40240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 -- 172.21.15.97:0/1002178185 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f0f7d3250 con 0x557f0fa30800 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.358+0000 7f6500903700 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa25fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009b80 tx=0x7f64f40079e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d42a30b8eb3abbb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.359+0000 7f64f2ffd700 1 -- 172.21.15.97:0/1002178185 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400e070 con 0x557f0fa30800 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.359+0000 7f64f2ffd700 1 -- 172.21.15.97:0/1002178185 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64f400f040 con 0x557f0fa30800 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.359+0000 7f64f2ffd700 1 -- 172.21.15.97:0/1002178185 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4005280 con 0x557f0fa30800 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 -- 172.21.15.97:0/1002178185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 msgr2=0x557f0fa25fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa25fd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009b80 tx=0x7f64f40079e0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 -- 172.21.15.97:0/1002178185 shutdown_connections 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa30800 0x557f0fa25fd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa2f060 0x557f0fa2f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 --2- 172.21.15.97:0/1002178185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.360+0000 7f650e54fb80 1 -- 172.21.15.97:0/1002178185 >> 172.21.15.97:0/1002178185 conn(0x557f0f9a3260 msgr2=0x557f0fa4b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.361+0000 7f650e54fb80 1 -- 172.21.15.97:0/1002178185 shutdown_connections 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.361+0000 7f650e54fb80 1 -- 172.21.15.97:0/1002178185 wait complete. 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.361+0000 7f650e54fb80 1 -- start start 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 Processor -- start 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f060 0x557f0fa3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa30800 0x557f0fa3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0fa3fb40 con 0x557f0fa2f9c0 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0fa3fcb0 con 0x557f0fa30800 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0fa3fe70 con 0x557f0fa2f060 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f650e54fb80 10 client.0 _ll_register_callbacks cb 0x7f64e4099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f64f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57838/0 (socket says 172.21.15.97:57838) 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f64f3fff700 1 -- 172.21.15.97:0/3903492624 learned_addr learned my addr 172.21.15.97:0/3903492624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.362+0000 7f6500903700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa30800 0x557f0fa3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 -- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f060 msgr2=0x557f0fa3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f060 0x557f0fa3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 -- 172.21.15.97:0/3903492624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa30800 msgr2=0x557f0fa3f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa30800 0x557f0fa3f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 -- 172.21.15.97:0/3903492624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f40073e0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64f3fff700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8502304611581a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e40a23b0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64ec01e040 con 0x557f0fa2f9c0 2023-10-08T09:57:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f64e409e170 con 0x557f0fa2f9c0 2023-10-08T09:57:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e40a1f20 con 0x557f0fa2f9c0 2023-10-08T09:57:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f650e54fb80 10 client.15969 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.363+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557f0f7ab2b0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64ec01e040 con 0x557f0fa2f9c0 2023-10-08T09:57:01.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f64e401a9d0 con 0x557f0fa2f9c0 2023-10-08T09:57:01.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 10 client.15969 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 10 client.15969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 20 client.15969 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa99780 0x557f0fa9bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 4 client.15969 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.364+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f0f97ab30 con 0x557f0fa99780 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.365+0000 7f64fbfff700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa99780 0x557f0fa9bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f64db7fe700 10 client.15969 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f64fbfff700 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa99780 0x557f0fa9bba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57819bc11e2c2925 server_cookie=591638529f4022f9 in_seq=0 out_seq=0 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f64db7fe700 1 -- 172.21.15.97:0/3903492624 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f0f97ab30 con 0x557f0fa99780 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f64db7fe700 10 client.15969 handle_command_reply: tid=0 2023-10-08T09:57:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 client.15969 shutdown 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 msgr2=0x557f0fa3c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e400eef0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 shutdown_connections 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x557f0fa99780 0x557f0fa9bba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f0fa30800 0x557f0fa3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f0fa2f060 0x557f0fa3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f64db7fe700 0 client.15969 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 --2- 172.21.15.97:0/3903492624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0fa2f9c0 0x557f0fa3c680 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.366+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 >> 172.21.15.97:0/3903492624 conn(0x557f0f9a3260 msgr2=0x557f0fa36cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.367+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 shutdown_connections 2023-10-08T09:57:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.367+0000 7f650e54fb80 1 -- 172.21.15.97:0/3903492624 wait complete. 2023-10-08T09:57:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.368+0000 7f650e54fb80 20 client.15969 trim_cache size 0 max 16384 2023-10-08T09:57:01.371 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.370+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 msgr2=0x7f64e80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.370+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 0x7f64e80a3a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec003720 comp rx=0 tx=0).stop 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.370+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 msgr2=0x7f64fc194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.370+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64e400e9a0 tx=0x7f64e400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 shutdown_connections 2023-10-08T09:57:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f64e80a15d0 0x7f64e80a3a90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64fc102350 0x7f64fc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64fc1007c0 0x7f64fc194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 --2- 172.21.15.97:0/660878849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc1019b0 0x7f64fc194b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.372+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 >> 172.21.15.97:0/660878849 conn(0x7f64fc0fbd30 msgr2=0x7f64fc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.374+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 shutdown_connections 2023-10-08T09:57:01.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.374+0000 7f6502366700 1 -- 172.21.15.97:0/660878849 wait complete. 2023-10-08T09:57:01.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T09:57:01.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 -- 172.21.15.97:0/1196074238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 msgr2=0x7f1e380fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 --2- 172.21.15.97:0/1196074238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e380fec80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0099c0 tx=0x7f1e2c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 -- 172.21.15.97:0/1196074238 shutdown_connections 2023-10-08T09:57:01.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 --2- 172.21.15.97:0/1196074238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e380ff1c0 0x7f1e38108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 --2- 172.21.15.97:0/1196074238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e38101db0 0x7f1e381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.517+0000 7f1e3d609700 1 --2- 172.21.15.97:0/1196074238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e380fec80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.518+0000 7f1e3d609700 1 -- 172.21.15.97:0/1196074238 >> 172.21.15.97:0/1196074238 conn(0x7f1e380fa3c0 msgr2=0x7f1e380fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.518+0000 7f1e3d609700 1 -- 172.21.15.97:0/1196074238 shutdown_connections 2023-10-08T09:57:01.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.518+0000 7f1e3d609700 1 -- 172.21.15.97:0/1196074238 wait complete. 2023-10-08T09:57:01.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.518+0000 7f1e3d609700 1 Processor -- start 2023-10-08T09:57:01.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.518+0000 7f1e3d609700 1 -- start start 2023-10-08T09:57:01.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e380ff1c0 0x7f1e38196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e38101db0 0x7f1e3819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3819e290 con 0x7f1e380fe820 2023-10-08T09:57:01.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3819e3d0 con 0x7f1e38101db0 2023-10-08T09:57:01.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e3d609700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3819e560 con 0x7f1e380ff1c0 2023-10-08T09:57:01.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57850/0 (socket says 172.21.15.97:57850) 2023-10-08T09:57:01.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 -- 172.21.15.97:0/992047709 learned_addr learned my addr 172.21.15.97:0/992047709 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e367fc700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e380ff1c0 0x7f1e38196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e377fe700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e38101db0 0x7f1e3819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 -- 172.21.15.97:0/992047709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e380ff1c0 msgr2=0x7f1e38196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e380ff1c0 0x7f1e38196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 -- 172.21.15.97:0/992047709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e38101db0 msgr2=0x7f1e3819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e38101db0 0x7f1e3819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 -- 172.21.15.97:0/992047709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0095f0 con 0x7f1e380fe820 2023-10-08T09:57:01.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c970 tx=0x7f1e2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.519+0000 7f1e1ffff700 1 -- 172.21.15.97:0/992047709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28004220 con 0x7f1e380fe820 2023-10-08T09:57:01.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.520+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3819e840 con 0x7f1e380fe820 2023-10-08T09:57:01.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.520+0000 7f1e1ffff700 1 -- 172.21.15.97:0/992047709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e28007d60 con 0x7f1e380fe820 2023-10-08T09:57:01.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.520+0000 7f1e1ffff700 1 -- 172.21.15.97:0/992047709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003c90 con 0x7f1e380fe820 2023-10-08T09:57:01.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.520+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e380ff930 con 0x7f1e380fe820 2023-10-08T09:57:01.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:01 smithi186 ceph-mon[104393]: pgmap v356: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-08T09:57:01.529 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1627740251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:57:01.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.522+0000 7f1e1ffff700 1 -- 172.21.15.97:0/992047709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1e28004380 con 0x7f1e380fe820 2023-10-08T09:57:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.522+0000 7f1e1ffff700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 0x7f1e200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e1ffff700 1 -- 172.21.15.97:0/992047709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e280ccbf0 con 0x7f1e380fe820 2023-10-08T09:57:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e367fc700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 0x7f1e200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e367fc700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 0x7f1e200a3940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009990 tx=0x7f1e2c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 Processor -- start 2023-10-08T09:57:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 -- start start 2023-10-08T09:57:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea9c0 0x5562d9aeb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aeb800 0x5562d9aebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9a52460 con 0x5562d9aea060 2023-10-08T09:57:01.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9a43c20 con 0x5562d9aeb800 2023-10-08T09:57:01.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.523+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9a41540 con 0x5562d9aea9c0 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57858/0 (socket says 172.21.15.97:57858) 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2187357479 learned_addr learned my addr 172.21.15.97:0/2187357479 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e367fc700 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea9c0 0x5562d9aeb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2187357479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea9c0 msgr2=0x5562d9aeb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea9c0 0x5562d9aeb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2187357479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aeb800 msgr2=0x5562d9aebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e377fe700 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aeb800 0x5562d9aebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aeb800 0x5562d9aebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2187357479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d9867de0 con 0x5562d9aea060 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1e28099b70 tx=0x7f1e28003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a231930c42e00f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e1d7fa700 1 -- 172.21.15.97:0/2187357479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28099be0 con 0x5562d9aea060 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e1d7fa700 1 -- 172.21.15.97:0/2187357479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2809dd30 con 0x5562d9aea060 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.524+0000 7f1e1d7fa700 1 -- 172.21.15.97:0/2187357479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2809a0a0 con 0x5562d9aea060 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2187357479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 msgr2=0x5562d9aea480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1e28099b70 tx=0x7f1e28003730 comp rx=0 tx=0).stop 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2187357479 shutdown_connections 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aeb800 0x5562d9aebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9aea480 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2187357479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea9c0 0x5562d9aeb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2187357479 >> 172.21.15.97:0/2187357479 conn(0x5562d9a5e260 msgr2=0x5562d9ae4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2187357479 shutdown_connections 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2187357479 wait complete. 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- start start 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 Processor -- start 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9afb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9a3ce30 con 0x5562d9aea060 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d99d9710 con 0x5562d9aea9c0 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d99d93b0 con 0x5562d9aeb800 2023-10-08T09:57:01.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.525+0000 7f1e497f2b80 10 client.0 _ll_register_callbacks cb 0x5562d9836800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57864/0 (socket says 172.21.15.97:57864) 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2880700703 learned_addr learned my addr 172.21.15.97:0/2880700703 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e377fe700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9afb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e367fc700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2880700703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 msgr2=0x5562d9afb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9afb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 msgr2=0x5562d9af4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 -- 172.21.15.97:0/2880700703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e380ff930 con 0x5562d9aea060 2023-10-08T09:57:01.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e28096650 tx=0x7f1e2809a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314a6e03d5d08f04 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28097d20 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e28099be0 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562d9a04eb0 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e280a1cf0 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e497f2b80 10 client.15981 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5562d9850c60 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562d9a04eb0 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1e28061d20 con 0x5562d9aea060 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.526+0000 7f1e497f2b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:01.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e497f2b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e497f2b80 20 client.15981 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9aff780 0x5562d9b01ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e497f2b80 4 client.15981 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562d9a35e10 con 0x5562d9aff780 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e367fc700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9aff780 0x5562d9b01ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e1aff5700 10 client.15981 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.527+0000 7f1e367fc700 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9aff780 0x5562d9b01ba0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf02a98e33ce3c server_cookie=37bb84be11225563 in_seq=0 out_seq=0 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e1aff5700 1 -- 172.21.15.97:0/2880700703 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562d9a35e10 con 0x5562d9aff780 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e1aff5700 10 client.15981 handle_command_reply: tid=0 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 client.15981 shutdown 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 msgr2=0x5562d9af3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e28096650 tx=0x7f1e2809a120 comp rx=0 tx=0).stop 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 shutdown_connections 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9aff780 0x5562d9b01ba0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9afb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e1aff5700 0 client.15981 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9af3ec0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/2880700703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 >> 172.21.15.97:0/2880700703 conn(0x5562d9a5e260 msgr2=0x5562d9ae44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 shutdown_connections 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 1 -- 172.21.15.97:0/2880700703 wait complete. 2023-10-08T09:57:01.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.528+0000 7f1e497f2b80 20 client.15981 trim_cache size 0 max 16384 2023-10-08T09:57:01.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 Processor -- start 2023-10-08T09:57:01.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 -- start start 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9afb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9ae4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d98bd920 con 0x5562d9aea060 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9a04eb0 con 0x5562d9aea9c0 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9afc570 con 0x5562d9aeb800 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57872/0 (socket says 172.21.15.97:57872) 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 -- 172.21.15.97:0/3277307391 learned_addr learned my addr 172.21.15.97:0/3277307391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e377fe700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9ae4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e367fc700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9afb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 -- 172.21.15.97:0/3277307391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 msgr2=0x5562d9ae4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9ae4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 -- 172.21.15.97:0/3277307391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 msgr2=0x5562d9afb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9afb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 -- 172.21.15.97:0/3277307391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d988e250 con 0x5562d9aea060 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e367fc700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9afb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.541+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e28099b70 tx=0x7f1e28003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa68add9f368ff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e1dffb700 1 -- 172.21.15.97:0/3277307391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28098060 con 0x5562d9aea060 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e1dffb700 1 -- 172.21.15.97:0/3277307391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e28095f20 con 0x5562d9aea060 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e1dffb700 1 -- 172.21.15.97:0/3277307391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28097e00 con 0x5562d9aea060 2023-10-08T09:57:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- 172.21.15.97:0/3277307391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 msgr2=0x5562d9afbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e28099b70 tx=0x7f1e28003730 comp rx=0 tx=0).stop 2023-10-08T09:57:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- 172.21.15.97:0/3277307391 shutdown_connections 2023-10-08T09:57:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aeb800 0x5562d9ae4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aea060 0x5562d9afbaa0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/3277307391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9afb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- 172.21.15.97:0/3277307391 >> 172.21.15.97:0/3277307391 conn(0x5562d9a5e260 msgr2=0x5562d9b064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- 172.21.15.97:0/3277307391 shutdown_connections 2023-10-08T09:57:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- 172.21.15.97:0/3277307391 wait complete. 2023-10-08T09:57:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 -- start start 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.542+0000 7f1e497f2b80 1 Processor -- start 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9afa930 con 0x5562d9aeb800 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9afaaa0 con 0x5562d9aea9c0 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d9afac60 con 0x5562d9aea060 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e497f2b80 10 client.0 _ll_register_callbacks cb 0x7f1e28099a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57882/0 (socket says 172.21.15.97:57882) 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e36ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59578/0 (socket says 172.21.15.97:59578) 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 -- 172.21.15.97:0/1234989521 learned_addr learned my addr 172.21.15.97:0/1234989521 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e367fc700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 -- 172.21.15.97:0/1234989521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 msgr2=0x5562d9af8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 -- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 msgr2=0x5562d9af9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 -- 172.21.15.97:0/1234989521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562d9850c60 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e377fe700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1e30008860 tx=0x7f1e30008b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e301f01f9dbafe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30018870 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.543+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e300189d0 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e2809dd30 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e30018b90 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 10 client.15987 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5562d98662b0 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2809dd30 con 0x5562d9aeb800 2023-10-08T09:57:01.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1e300173c0 con 0x5562d9aeb800 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 20 client.15987 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9b543a0 0x5562d9b567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 4 client.15987 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5562d9a35b30 con 0x5562d9b543a0 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.544+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9b543a0 0x5562d9b567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e1aff5700 10 client.15987 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:01.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e36ffd700 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9b543a0 0x5562d9b567c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dcf84c78f25986 server_cookie=a9c976cbac0934ac in_seq=0 out_seq=0 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e1aff5700 1 -- 172.21.15.97:0/1234989521 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5562d9a35b30 con 0x5562d9b543a0 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e1aff5700 10 client.15987 handle_command_reply: tid=0 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 client.15987 shutdown 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 msgr2=0x5562d9afa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1e30008860 tx=0x7f1e30008b70 comp rx=0 tx=0).stop 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 shutdown_connections 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5562d9aeb800 0x5562d9afa3f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5562d9b543a0 0x5562d9b567c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562d9aea060 0x5562d9af8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e1aff5700 0 client.15987 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:01.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 --2- 172.21.15.97:0/1234989521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5562d9aea9c0 0x5562d9af9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 >> 172.21.15.97:0/1234989521 conn(0x5562d9a5e260 msgr2=0x5562d9af33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 shutdown_connections 2023-10-08T09:57:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 1 -- 172.21.15.97:0/1234989521 wait complete. 2023-10-08T09:57:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e497f2b80 20 client.15987 trim_cache size 0 max 16384 2023-10-08T09:57:01.690 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:57:01.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 msgr2=0x7f1e200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 0x7f1e200a3940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009990 tx=0x7f1e2c005ec0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 msgr2=0x7f1e381967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c970 tx=0x7f1e2800cc80 comp rx=0 tx=0).stop 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 shutdown_connections 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f1e200a1480 0x7f1e200a3940 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e38101db0 0x7f1e3819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e380fe820 0x7f1e381967b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 --2- 172.21.15.97:0/992047709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e380ff1c0 0x7f1e38196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.545+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 >> 172.21.15.97:0/992047709 conn(0x7f1e380fa3c0 msgr2=0x7f1e38104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.546+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 shutdown_connections 2023-10-08T09:57:01.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:01.546+0000 7f1e3d609700 1 -- 172.21.15.97:0/992047709 wait complete. 2023-10-08T09:57:01.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:01 smithi097 ceph-mon[99491]: pgmap v356: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-08T09:57:01.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1627740251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:57:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:01 smithi178 ceph-mon[103156]: pgmap v356: 129 pgs: 129 active+clean; 708 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-08T09:57:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1627740251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:57:01.795 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether build environment is sane... yes 2023-10-08T09:57:01.799 INFO:tasks.workunit.client.0.smithi097.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-10-08T09:57:01.800 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gawk... gawk 2023-10-08T09:57:01.812 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether make sets $(MAKE)... yes 2023-10-08T09:57:01.843 INFO:tasks.workunit.client.0.smithi097.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-08T09:57:01.844 INFO:tasks.workunit.client.0.smithi097.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-08T09:57:01.854 INFO:tasks.workunit.client.0.smithi097.stdout:checking for style of include used by make... GNU 2023-10-08T09:57:01.857 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc... gcc 2023-10-08T09:57:01.930 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether the C compiler works... yes 2023-10-08T09:57:01.930 INFO:tasks.workunit.client.0.smithi097.stdout:checking for C compiler default output file name... a.out 2023-10-08T09:57:01.973 INFO:tasks.workunit.client.0.smithi097.stdout:checking for suffix of executables... 2023-10-08T09:57:02.009 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are cross compiling... no 2023-10-08T09:57:02.050 INFO:tasks.workunit.client.0.smithi097.stdout:checking for suffix of object files... o 2023-10-08T09:57:02.075 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are using the GNU C compiler... yes 2023-10-08T09:57:02.097 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether gcc accepts -g... yes 2023-10-08T09:57:02.129 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc option to accept ISO C89... none needed 2023-10-08T09:57:02.217 INFO:tasks.workunit.client.0.smithi097.stdout:checking dependency style of gcc... gcc3 2023-10-08T09:57:02.256 INFO:tasks.workunit.client.0.smithi097.stdout:checking how to run the C preprocessor... gcc -E 2023-10-08T09:57:02.300 INFO:tasks.workunit.client.0.smithi097.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-10-08T09:57:02.301 INFO:tasks.workunit.client.0.smithi097.stdout:checking for egrep... /usr/bin/grep -E 2023-10-08T09:57:02.423 INFO:tasks.workunit.client.0.smithi097.stdout:checking for ANSI C header files... yes 2023-10-08T09:57:02.454 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/types.h... yes 2023-10-08T09:57:02.492 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/stat.h... yes 2023-10-08T09:57:02.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:02 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:02.528 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdlib.h... yes 2023-10-08T09:57:02.567 INFO:tasks.workunit.client.0.smithi097.stdout:checking for string.h... yes 2023-10-08T09:57:02.604 INFO:tasks.workunit.client.0.smithi097.stdout:checking for memory.h... yes 2023-10-08T09:57:02.643 INFO:tasks.workunit.client.0.smithi097.stdout:checking for strings.h... yes 2023-10-08T09:57:02.681 INFO:tasks.workunit.client.0.smithi097.stdout:checking for inttypes.h... yes 2023-10-08T09:57:02.694 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:57:02.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:57:02.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:02 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:02.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:02 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:02.749 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdint.h... yes 2023-10-08T09:57:02.788 INFO:tasks.workunit.client.0.smithi097.stdout:checking for unistd.h... yes 2023-10-08T09:57:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 -- 172.21.15.97:0/1436596822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40101230 msgr2=0x7fdf40107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 --2- 172.21.15.97:0/1436596822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40101230 0x7fdf40107ae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0099c0 tx=0x7fdf3c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 -- 172.21.15.97:0/1436596822 shutdown_connections 2023-10-08T09:57:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 --2- 172.21.15.97:0/1436596822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40101230 0x7fdf40107ae0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 --2- 172.21.15.97:0/1436596822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf400ff6a0 0x7fdf400ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 --2- 172.21.15.97:0/1436596822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf40100890 0x7fdf40100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 -- 172.21.15.97:0/1436596822 >> 172.21.15.97:0/1436596822 conn(0x7fdf400fac10 msgr2=0x7fdf400fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 -- 172.21.15.97:0/1436596822 shutdown_connections 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.823+0000 7fdf486a6700 1 -- 172.21.15.97:0/1436596822 wait complete. 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 Processor -- start 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 -- start start 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 0x7fdf401035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40102950 con 0x7fdf40100890 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40102ac0 con 0x7fdf40101230 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf486a6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf401a0720 con 0x7fdf400ff6a0 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf46c43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57894/0 (socket says 172.21.15.97:57894) 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf46c43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37866/0 (socket says 172.21.15.97:37866) 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 -- 172.21.15.97:0/4285556114 learned_addr learned my addr 172.21.15.97:0/4285556114 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf46442700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 0x7fdf401035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 -- 172.21.15.97:0/4285556114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 msgr2=0x7fdf401035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 0x7fdf401035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 -- 172.21.15.97:0/4285556114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 msgr2=0x7fdf401021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf45c41700 1 -- 172.21.15.97:0/4285556114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0095f0 con 0x7fdf40100890 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf46442700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 0x7fdf401035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.824+0000 7fdf46c43700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf45c41700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c920 tx=0x7fdf3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf377fe700 1 -- 172.21.15.97:0/4285556114 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30007aa0 con 0x7fdf40100890 2023-10-08T09:57:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf401a09b0 con 0x7fdf40100890 2023-10-08T09:57:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf377fe700 1 -- 172.21.15.97:0/4285556114 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3000ce70 con 0x7fdf40100890 2023-10-08T09:57:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf377fe700 1 -- 172.21.15.97:0/4285556114 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30005440 con 0x7fdf40100890 2023-10-08T09:57:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.825+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf401a0ec0 con 0x7fdf40100890 2023-10-08T09:57:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.827+0000 7fdf377fe700 1 -- 172.21.15.97:0/4285556114 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdf30004750 con 0x7fdf40100890 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.827+0000 7fdf377fe700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 0x7fdf2c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.828+0000 7fdf46442700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 0x7fdf2c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.828+0000 7fdf377fe700 1 -- 172.21.15.97:0/4285556114 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdf300ccc10 con 0x7fdf40100890 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.828+0000 7fdf46442700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 0x7fdf2c0a3940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf401006f0 tx=0x7fdf3800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.828+0000 7fdf5488fb80 1 Processor -- start 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 -- start start 2023-10-08T09:57:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdd9a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9b880 0x55f6cdd9bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdd024e0 con 0x55f6cdd9aa40 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdcf3ca0 con 0x55f6cdd9b880 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdcf15c0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57902/0 (socket says 172.21.15.97:57902) 2023-10-08T09:57:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 -- 172.21.15.97:0/965717443 learned_addr learned my addr 172.21.15.97:0/965717443 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf46c43700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9b880 0x55f6cdd9bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf46442700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdd9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 -- 172.21.15.97:0/965717443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 msgr2=0x55f6cdd9a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdd9a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 -- 172.21.15.97:0/965717443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9b880 msgr2=0x55f6cdd9bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9b880 0x55f6cdd9bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 -- 172.21.15.97:0/965717443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6cdb17e60 con 0x55f6cdd9aa40 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf46442700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdd9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.829+0000 7fdf45c41700 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdf30099b90 tx=0x7fdf30099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4247fbc67ea6a577 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf34ff9700 1 -- 172.21.15.97:0/965717443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30099c00 con 0x55f6cdd9aa40 2023-10-08T09:57:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf34ff9700 1 -- 172.21.15.97:0/965717443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3009dd50 con 0x55f6cdd9aa40 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf34ff9700 1 -- 172.21.15.97:0/965717443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3009a080 con 0x55f6cdd9aa40 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 -- 172.21.15.97:0/965717443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 msgr2=0x55f6cdd9b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdf30099b90 tx=0x7fdf30099560 comp rx=0 tx=0).stop 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 -- 172.21.15.97:0/965717443 shutdown_connections 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9b880 0x55f6cdd9bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdd9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/965717443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cdd9b2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 -- 172.21.15.97:0/965717443 >> 172.21.15.97:0/965717443 conn(0x55f6cdd0e2e0 msgr2=0x55f6cdd94e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 -- 172.21.15.97:0/965717443 shutdown_connections 2023-10-08T09:57:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.830+0000 7fdf5488fb80 1 -- 172.21.15.97:0/965717443 wait complete. 2023-10-08T09:57:02.839 INFO:tasks.workunit.client.0.smithi097.stdout:checking minix/config.h usability... no 2023-10-08T09:57:02.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 -- start start 2023-10-08T09:57:02.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 Processor -- start 2023-10-08T09:57:02.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cddab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdceceb0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdc89790 con 0x55f6cdd9aa40 2023-10-08T09:57:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdc89430 con 0x55f6cdd9b880 2023-10-08T09:57:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf5488fb80 10 client.0 _ll_register_callbacks cb 0x55f6cdae6880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57906/0 (socket says 172.21.15.97:57906) 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 -- 172.21.15.97:0/190920167 learned_addr learned my addr 172.21.15.97:0/190920167 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf45c41700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46c43700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cddab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 -- 172.21.15.97:0/190920167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 msgr2=0x55f6cddab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cddab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 -- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9aa40 msgr2=0x55f6cdda43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46442700 1 -- 172.21.15.97:0/190920167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf401a0ec0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.831+0000 7fdf46c43700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cddab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf46442700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdf38016a50 tx=0x7fdf38004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436dd534b1ebf8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38007e90 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6cdcb4f30 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdf38016e10 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf38014a70 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 10 client.15999 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f6cdb00ce0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6cdcb4f30 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdf38022c70 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 20 client.15999 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cddafda0 0x55f6cddb21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 4 client.15999 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.832+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6cdce5e90 con 0x55f6cddafda0 2023-10-08T09:57:02.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.833+0000 7fdf45c41700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cddafda0 0x55f6cddb21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.833+0000 7fdf45c41700 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cddafda0 0x55f6cddb21c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7bb7d5b4d6ba4a server_cookie=86385c4a2386e09c in_seq=0 out_seq=0 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.833+0000 7fdf29ffb700 10 client.15999 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.833+0000 7fdf29ffb700 1 -- 172.21.15.97:0/190920167 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6cdce5e90 con 0x55f6cddafda0 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.833+0000 7fdf29ffb700 10 client.15999 handle_command_reply: tid=0 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 client.15999 shutdown 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 msgr2=0x55f6cdda3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdf38016a50 tx=0x7fdf38004b40 comp rx=0 tx=0).stop 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 shutdown_connections 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cddafda0 0x55f6cddb21c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cddab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3e90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf29ffb700 0 client.15999 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/190920167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 >> 172.21.15.97:0/190920167 conn(0x55f6cdd0e2e0 msgr2=0x55f6cdd94550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 shutdown_connections 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 1 -- 172.21.15.97:0/190920167 wait complete. 2023-10-08T09:57:02.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.834+0000 7fdf5488fb80 20 client.15999 trim_cache size 0 max 16384 2023-10-08T09:57:02.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 Processor -- start 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 -- start start 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cdda35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdb6d9a0 con 0x55f6cdd9aa40 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdd91b60 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.851+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdda4b10 con 0x55f6cdd9b880 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf46442700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57916/0 (socket says 172.21.15.97:57916) 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 -- 172.21.15.97:0/1023695505 learned_addr learned my addr 172.21.15.97:0/1023695505 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf46c43700 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cdda35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 -- 172.21.15.97:0/1023695505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 msgr2=0x55f6cdda35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cdda35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 -- 172.21.15.97:0/1023695505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 msgr2=0x55f6cdda3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 -- 172.21.15.97:0/1023695505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6cdd91320 con 0x55f6cdd9aa40 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf45c41700 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c8f0 tx=0x7fdf30007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953959cafb893a4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf357fa700 1 -- 172.21.15.97:0/1023695505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30099c00 con 0x55f6cdd9aa40 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf357fa700 1 -- 172.21.15.97:0/1023695505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf30097d40 con 0x55f6cdd9aa40 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.852+0000 7fdf357fa700 1 -- 172.21.15.97:0/1023695505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300990f0 con 0x55f6cdd9aa40 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- 172.21.15.97:0/1023695505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 msgr2=0x55f6cddab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c8f0 tx=0x7fdf30007e80 comp rx=0 tx=0).stop 2023-10-08T09:57:02.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- 172.21.15.97:0/1023695505 shutdown_connections 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9b880 0x55f6cdda35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cdda3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/1023695505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9aa40 0x55f6cddab380 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- 172.21.15.97:0/1023695505 >> 172.21.15.97:0/1023695505 conn(0x55f6cdd0e2e0 msgr2=0x55f6cdd9e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- 172.21.15.97:0/1023695505 shutdown_connections 2023-10-08T09:57:02.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- 172.21.15.97:0/1023695505 wait complete. 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 -- start start 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.853+0000 7fdf5488fb80 1 Processor -- start 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9b880 0x55f6cdda7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cddab750 con 0x55f6cdd9b880 2023-10-08T09:57:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdda8450 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cdda8610 con 0x55f6cdd9aa40 2023-10-08T09:57:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 10 client.0 _ll_register_callbacks cb 0x7fdf38007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.862 INFO:tasks.workunit.client.0.smithi097.stdout:checking minix/config.h presence... no 2023-10-08T09:57:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37890/0 (socket says 172.21.15.97:37890) 2023-10-08T09:57:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 -- 172.21.15.97:0/3379009055 learned_addr learned my addr 172.21.15.97:0/3379009055 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46c43700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9b880 0x55f6cdda7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf45c41700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.863 INFO:tasks.workunit.client.0.smithi097.stdout:checking for minix/config.h... no 2023-10-08T09:57:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 -- 172.21.15.97:0/3379009055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9aa40 msgr2=0x55f6cdda79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 -- 172.21.15.97:0/3379009055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9b880 msgr2=0x55f6cdda7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9b880 0x55f6cdda7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 -- 172.21.15.97:0/3379009055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6cdb17e60 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf46442700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf3802b480 tx=0x7fdf38007fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4103fe4a5648a8ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-08T09:57:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380146b0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf3009dd50 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdf38014ea0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf38007bf0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 10 client.25433 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.854+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f6cdb3e2d0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3009dd50 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdf380228e0 con 0x55f6cdd9a0e0 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 10 client.25433 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 20 client.25433 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cde05560 0x55f6cde07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 4 client.25433 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f6cdce5bb0 con 0x55f6cde05560 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf45c41700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cde05560 0x55f6cde07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf29ffb700 10 client.25433 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:02.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.855+0000 7fdf45c41700 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cde05560 0x55f6cde07980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccb8bb2bb55c997 server_cookie=d49baeb91f6c970c in_seq=0 out_seq=0 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.857+0000 7fdf29ffb700 1 -- 172.21.15.97:0/3379009055 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f6cdce5bb0 con 0x55f6cde05560 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.857+0000 7fdf29ffb700 10 client.25433 handle_command_reply: tid=0 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.857+0000 7fdf5488fb80 1 client.25433 shutdown 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 msgr2=0x55f6cddaaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf3802b480 tx=0x7fdf38007fd0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 shutdown_connections 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f6cdd9b880 0x55f6cdda7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6cdd9a0e0 0x55f6cddaaf30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f6cde05560 0x55f6cde07980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 --2- 172.21.15.97:0/3379009055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6cdd9aa40 0x55f6cdda79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf29ffb700 0 client.25433 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 >> 172.21.15.97:0/3379009055 conn(0x55f6cdd0e2e0 msgr2=0x55f6cdd9e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 shutdown_connections 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 1 -- 172.21.15.97:0/3379009055 wait complete. 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf5488fb80 20 client.25433 trim_cache size 0 max 16384 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "bfbcb446-9fc8-4289-8b54-7b11acd13609", 2023-10-08T09:57:02.872 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2023-10-08T09:57:02.873 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 msgr2=0x7fdf2c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 0x7fdf2c0a3940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf401006f0 tx=0x7fdf3800b040 comp rx=0 tx=0).stop 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 msgr2=0x7fdf40101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c920 tx=0x7fdf3000cc30 comp rx=0 tx=0).stop 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 shutdown_connections 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdf2c0a1480 0x7fdf2c0a3940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf40101230 0x7fdf401021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf400ff6a0 0x7fdf401035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 --2- 172.21.15.97:0/4285556114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf40100890 0x7fdf40101c70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:02.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 >> 172.21.15.97:0/4285556114 conn(0x7fdf400fac10 msgr2=0x7fdf400fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:02.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.858+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 shutdown_connections 2023-10-08T09:57:02.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:02.859+0000 7fdf486a6700 1 -- 172.21.15.97:0/4285556114 wait complete. 2023-10-08T09:57:02.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:57:02.896 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2023-10-08T09:57:02.896 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc... (cached) gcc 2023-10-08T09:57:02.927 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-10-08T09:57:02.928 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether gcc accepts -g... (cached) yes 2023-10-08T09:57:02.928 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-10-08T09:57:02.928 INFO:tasks.workunit.client.0.smithi097.stdout:checking dependency style of gcc... (cached) gcc3 2023-10-08T09:57:02.970 INFO:tasks.workunit.client.0.smithi097.stdout:checking for main in -lm... yes 2023-10-08T09:57:03.012 INFO:tasks.workunit.client.0.smithi097.stdout:checking for main in -lpthread... yes 2023-10-08T09:57:03.013 INFO:tasks.workunit.client.0.smithi097.stdout:checking for ANSI C header files... (cached) yes 2023-10-08T09:57:03.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 -- 172.21.15.97:0/538610126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 msgr2=0x7fda64107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/538610126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda64107ae0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b0d0 tx=0x7fda5c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 -- 172.21.15.97:0/538610126 shutdown_connections 2023-10-08T09:57:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/538610126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda64107ae0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/538610126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda640ff6a0 0x7fda640ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/538610126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda64100890 0x7fda64100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 -- 172.21.15.97:0/538610126 >> 172.21.15.97:0/538610126 conn(0x7fda640fac10 msgr2=0x7fda640fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 -- 172.21.15.97:0/538610126 shutdown_connections 2023-10-08T09:57:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 -- 172.21.15.97:0/538610126 wait complete. 2023-10-08T09:57:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.026+0000 7fda6ac9b700 1 Processor -- start 2023-10-08T09:57:03.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 -- start start 2023-10-08T09:57:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda640ff6a0 0x7fda64198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda64100890 0x7fda64198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a05c0 con 0x7fda64101230 2023-10-08T09:57:03.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a0700 con 0x7fda640ff6a0 2023-10-08T09:57:03.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda6ac9b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a0890 con 0x7fda64100890 2023-10-08T09:57:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57938/0 (socket says 172.21.15.97:57938) 2023-10-08T09:57:03.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 -- 172.21.15.97:0/1035256561 learned_addr learned my addr 172.21.15.97:0/1035256561 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda68a37700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda640ff6a0 0x7fda64198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda63fff700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda64100890 0x7fda64198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 -- 172.21.15.97:0/1035256561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda64100890 msgr2=0x7fda64198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda64100890 0x7fda64198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 -- 172.21.15.97:0/1035256561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda640ff6a0 msgr2=0x7fda64198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda640ff6a0 0x7fda64198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 -- 172.21.15.97:0/1035256561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009d00 con 0x7fda64101230 2023-10-08T09:57:03.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda69238700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007bd0 tx=0x7fda5c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.027+0000 7fda61ffb700 1 -- 172.21.15.97:0/1035256561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00e040 con 0x7fda64101230 2023-10-08T09:57:03.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.028+0000 7fda61ffb700 1 -- 172.21.15.97:0/1035256561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda5c01dd50 con 0x7fda64101230 2023-10-08T09:57:03.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.028+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda641a0b10 con 0x7fda64101230 2023-10-08T09:57:03.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.028+0000 7fda61ffb700 1 -- 172.21.15.97:0/1035256561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0123f0 con 0x7fda64101230 2023-10-08T09:57:03.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.028+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641a0f90 con 0x7fda64101230 2023-10-08T09:57:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.030+0000 7fda61ffb700 1 -- 172.21.15.97:0/1035256561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fda5c01b550 con 0x7fda64101230 2023-10-08T09:57:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda61ffb700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 0x7fda540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda68a37700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 0x7fda540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda61ffb700 1 -- 172.21.15.97:0/1035256561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda5c0cee70 con 0x7fda64101230 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda68a37700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 0x7fda540a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda641006f0 tx=0x7fda5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda76e84b80 1 Processor -- start 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda76e84b80 1 -- start start 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a3060 0x55f3428a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.031+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a39c0 0x55f3428a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda76e84b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f34280b460 con 0x55f3428a4800 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda76e84b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3427fcc20 con 0x55f3428a39c0 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda76e84b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3427fa540 con 0x55f3428a3060 2023-10-08T09:57:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57946/0 (socket says 172.21.15.97:57946) 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 -- 172.21.15.97:0/214804060 learned_addr learned my addr 172.21.15.97:0/214804060 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda68a37700 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a3060 0x55f3428a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda63fff700 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a39c0 0x55f3428a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 -- 172.21.15.97:0/214804060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a3060 msgr2=0x55f3428a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a3060 0x55f3428a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 -- 172.21.15.97:0/214804060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a39c0 msgr2=0x55f3428a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a39c0 0x55f3428a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 -- 172.21.15.97:0/214804060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f342620de0 con 0x55f3428a4800 2023-10-08T09:57:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda69238700 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009fd0 tx=0x7fda5c01b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50118bb01f91dd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda4effd700 1 -- 172.21.15.97:0/214804060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c09b850 con 0x55f3428a4800 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda4effd700 1 -- 172.21.15.97:0/214804060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda5c09c5f0 con 0x55f3428a4800 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.032+0000 7fda4effd700 1 -- 172.21.15.97:0/214804060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0a2190 con 0x55f3428a4800 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- 172.21.15.97:0/214804060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 msgr2=0x55f3428a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009fd0 tx=0x7fda5c01b950 comp rx=0 tx=0).stop 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- 172.21.15.97:0/214804060 shutdown_connections 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428a4c90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a3060 0x55f3428a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 --2- 172.21.15.97:0/214804060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a39c0 0x55f3428a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- 172.21.15.97:0/214804060 >> 172.21.15.97:0/214804060 conn(0x55f342817260 msgr2=0x55f34289de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- 172.21.15.97:0/214804060 shutdown_connections 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- 172.21.15.97:0/214804060 wait complete. 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 -- start start 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.033+0000 7fda76e84b80 1 Processor -- start 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3427f5e30 con 0x55f3428a3060 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f342792710 con 0x55f3428a4800 2023-10-08T09:57:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3427923b0 con 0x55f3428a39c0 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda76e84b80 10 client.0 _ll_register_callbacks cb 0x55f3425ef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57948/0 (socket says 172.21.15.97:57948) 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 -- 172.21.15.97:0/878423962 learned_addr learned my addr 172.21.15.97:0/878423962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda69238700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda63fff700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 -- 172.21.15.97:0/878423962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 msgr2=0x55f3428ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 -- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 msgr2=0x55f3428b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.034+0000 7fda68a37700 1 -- 172.21.15.97:0/878423962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641a0f90 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda68a37700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fda580116f0 tx=0x7fda580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7dcc8d5c0d5e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58010ee0 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda58017070 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3427bdeb0 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda58028470 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 10 client.16017 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f342609c60 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3427bdeb0 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda5801f800 con 0x55f3428a3060 2023-10-08T09:57:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 20 client.16017 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f3428b8970 0x55f3428bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 4 client.16017 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.035+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3427eee10 con 0x55f3428b8970 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.036+0000 7fda63fff700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f3428b8970 0x55f3428bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.036+0000 7fda43fff700 10 client.16017 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.036+0000 7fda63fff700 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f3428b8970 0x55f3428bad90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f4d7733b081f06 server_cookie=dc9bd7b2847023e0 in_seq=0 out_seq=0 2023-10-08T09:57:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda43fff700 1 -- 172.21.15.97:0/878423962 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3427eee10 con 0x55f3428b8970 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda43fff700 10 client.16017 handle_command_reply: tid=0 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 client.16017 shutdown 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 msgr2=0x55f3428acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fda580116f0 tx=0x7fda580047c0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 shutdown_connections 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428acb00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f3428b8970 0x55f3428bad90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 --2- 172.21.15.97:0/878423962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda43fff700 0 client.16017 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 >> 172.21.15.97:0/878423962 conn(0x55f342817260 msgr2=0x55f34289d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 shutdown_connections 2023-10-08T09:57:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 1 -- 172.21.15.97:0/878423962 wait complete. 2023-10-08T09:57:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.037+0000 7fda76e84b80 20 client.16017 trim_cache size 0 max 16384 2023-10-08T09:57:03.056 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-10-08T09:57:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 Processor -- start 2023-10-08T09:57:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 -- start start 2023-10-08T09:57:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f342676920 con 0x55f3428a3060 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3428a3e60 con 0x55f3428a4800 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda76e84b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3428b5d90 con 0x55f3428a39c0 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57956/0 (socket says 172.21.15.97:57956) 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 -- 172.21.15.97:0/2164617209 learned_addr learned my addr 172.21.15.97:0/2164617209 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda69238700 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 -- 172.21.15.97:0/2164617209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 msgr2=0x55f3428b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 -- 172.21.15.97:0/2164617209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 msgr2=0x55f3428a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 -- 172.21.15.97:0/2164617209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f342647250 con 0x55f3428a3060 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda68a37700 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fda580116c0 tx=0x7fda58005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9380b0937dbd86d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.055+0000 7fda4f7fe700 1 -- 172.21.15.97:0/2164617209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5801f630 con 0x55f3428a3060 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda4f7fe700 1 -- 172.21.15.97:0/2164617209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda58015070 con 0x55f3428a3060 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda4f7fe700 1 -- 172.21.15.97:0/2164617209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58028dd0 con 0x55f3428a3060 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 -- 172.21.15.97:0/2164617209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 msgr2=0x55f3428aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fda580116c0 tx=0x7fda58005d00 comp rx=0 tx=0).stop 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 -- 172.21.15.97:0/2164617209 shutdown_connections 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a4800 0x55f3428a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a3060 0x55f3428aadd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2164617209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 -- 172.21.15.97:0/2164617209 >> 172.21.15.97:0/2164617209 conn(0x55f342817260 msgr2=0x55f342899f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 -- 172.21.15.97:0/2164617209 shutdown_connections 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.056+0000 7fda76e84b80 1 -- 172.21.15.97:0/2164617209 wait complete. 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 -- start start 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 Processor -- start 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 0x55f3428b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3428b2770 con 0x55f3428a4800 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3428b28e0 con 0x55f3428a3060 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3428b2aa0 con 0x55f3428a39c0 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda76e84b80 10 client.0 _ll_register_callbacks cb 0x7fda58010d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57966/0 (socket says 172.21.15.97:57966) 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 -- 172.21.15.97:0/2414089315 learned_addr learned my addr 172.21.15.97:0/2414089315 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda68a37700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 0x55f3428b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda63fff700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 -- 172.21.15.97:0/2414089315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 msgr2=0x55f3428b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 -- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 msgr2=0x55f3428b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 0x55f3428b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda69238700 1 -- 172.21.15.97:0/2414089315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5802d040 con 0x55f3428a4800 2023-10-08T09:57:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.057+0000 7fda68a37700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 0x55f3428b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda69238700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0983f0 tx=0x7fda5c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf94946868e73169 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c09b850 con 0x55f3428a4800 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda5c0a41f0 con 0x55f3428a4800 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda5c09c5f0 con 0x55f3428a4800 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda5c0a0040 con 0x55f3428a4800 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 10 client.16023 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f34261f2b0 con 0x55f3428a4800 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda5c09c5f0 con 0x55f3428a4800 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda5c09b3e0 con 0x55f3428a4800 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 20 client.16023 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f34290d800 0x55f34290fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 4 client.16023 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:57:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3427eeb30 con 0x55f34290d800 2023-10-08T09:57:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.058+0000 7fda68a37700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f34290d800 0x55f34290fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.059+0000 7fda43fff700 10 client.16023 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.059+0000 7fda68a37700 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f34290d800 0x55f34290fc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4cf95da5db8fe8 server_cookie=9fe717077997ee3b in_seq=0 out_seq=0 2023-10-08T09:57:03.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.061+0000 7fda43fff700 1 -- 172.21.15.97:0/2414089315 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f3427eeb30 con 0x55f34290d800 2023-10-08T09:57:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.061+0000 7fda43fff700 10 client.16023 handle_command_reply: tid=0 2023-10-08T09:57:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.061+0000 7fda76e84b80 1 client.16023 shutdown 2023-10-08T09:57:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.061+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 msgr2=0x55f3428b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.061+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0983f0 tx=0x7fda5c009ba0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 shutdown_connections 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f3428a4800 0x55f3428b2060 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f3428a3060 0x55f3428b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f34290d800 0x55f34290fc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 --2- 172.21.15.97:0/2414089315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f3428a39c0 0x55f3428b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda43fff700 0 client.16023 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 >> 172.21.15.97:0/2414089315 conn(0x55f342817260 msgr2=0x55f3428a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 shutdown_connections 2023-10-08T09:57:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 1 -- 172.21.15.97:0/2414089315 wait complete. 2023-10-08T09:57:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda76e84b80 20 client.16023 trim_cache size 0 max 16384 2023-10-08T09:57:03.074 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bfbcb446-9fc8-4289-8b54-7b11acd13609": { 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "bfbcb446-9fc8-4289-8b54-7b11acd13609", 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:57:03.075 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 msgr2=0x7fda540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 0x7fda540a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda641006f0 tx=0x7fda5800b410 comp rx=0 tx=0).stop 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 msgr2=0x7fda6419fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007bd0 tx=0x7fda5c009440 comp rx=0 tx=0).stop 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 shutdown_connections 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fda540a1500 0x7fda540a39c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda64101230 0x7fda6419fff0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda640ff6a0 0x7fda64198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 --2- 172.21.15.97:0/1035256561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda64100890 0x7fda64198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 >> 172.21.15.97:0/1035256561 conn(0x7fda640fac10 msgr2=0x7fda640fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 shutdown_connections 2023-10-08T09:57:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:03.062+0000 7fda6ac9b700 1 -- 172.21.15.97:0/1035256561 wait complete. 2023-10-08T09:57:03.079 INFO:tasks.cephfs.filesystem:scrub status for tag:bfbcb446-9fc8-4289-8b54-7b11acd13609 - {'path': '/', 'tag': 'bfbcb446-9fc8-4289-8b54-7b11acd13609', 'options': 'recursive,force'} 2023-10-08T09:57:03.101 INFO:tasks.workunit.client.0.smithi097.stdout:checking pthread.h usability... yes 2023-10-08T09:57:03.119 INFO:tasks.workunit.client.0.smithi097.stdout:checking pthread.h presence... yes 2023-10-08T09:57:03.119 INFO:tasks.workunit.client.0.smithi097.stdout:checking for pthread.h... yes 2023-10-08T09:57:03.166 INFO:tasks.workunit.client.0.smithi097.stdout:checking fcntl.h usability... yes 2023-10-08T09:57:03.183 INFO:tasks.workunit.client.0.smithi097.stdout:checking fcntl.h presence... yes 2023-10-08T09:57:03.184 INFO:tasks.workunit.client.0.smithi097.stdout:checking for fcntl.h... yes 2023-10-08T09:57:03.230 INFO:tasks.workunit.client.0.smithi097.stdout:checking limits.h usability... yes 2023-10-08T09:57:03.243 INFO:tasks.workunit.client.0.smithi097.stdout:checking limits.h presence... yes 2023-10-08T09:57:03.243 INFO:tasks.workunit.client.0.smithi097.stdout:checking for limits.h... yes 2023-10-08T09:57:03.257 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stdint.h... (cached) yes 2023-10-08T09:57:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 -- 172.21.15.97:0/2184837273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0000100830 msgr2=0x7f0000100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 --2- 172.21.15.97:0/2184837273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0000100830 0x7f0000100c90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efff00099c0 tx=0x7efff0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 -- 172.21.15.97:0/2184837273 shutdown_connections 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 --2- 172.21.15.97:0/2184837273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00001011d0 0x7f0000107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 --2- 172.21.15.97:0/2184837273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f00000ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 --2- 172.21.15.97:0/2184837273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0000100830 0x7f0000100c90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 -- 172.21.15.97:0/2184837273 >> 172.21.15.97:0/2184837273 conn(0x7f00000fabb0 msgr2=0x7f00000fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 -- 172.21.15.97:0/2184837273 shutdown_connections 2023-10-08T09:57:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.273+0000 7f0007c40700 1 -- 172.21.15.97:0/2184837273 wait complete. 2023-10-08T09:57:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.274+0000 7f0007c40700 1 Processor -- start 2023-10-08T09:57:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.274+0000 7f0007c40700 1 -- start start 2023-10-08T09:57:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.274+0000 7f0007c40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f0000195510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.274+0000 7f0007c40700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0000100830 0x7f0000195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f0007c40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 0x7f000019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f0007c40700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000019d040 con 0x7f00001011d0 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f0007c40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000019d180 con 0x7f00000ff640 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f0007c40700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000019d310 con 0x7f0000100830 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00061dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 0x7f000019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f0000195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00061dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 0x7f000019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57980/0 (socket says 172.21.15.97:57980) 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00061dd700 1 -- 172.21.15.97:0/2357232950 learned_addr learned my addr 172.21.15.97:0/2357232950 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00051db700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0000100830 0x7f0000195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 -- 172.21.15.97:0/2357232950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0000100830 msgr2=0x7f0000195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0000100830 0x7f0000195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 -- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 msgr2=0x7f000019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 0x7f000019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 -- 172.21.15.97:0/2357232950 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00095f0 con 0x7f00000ff640 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f00059dc700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f0000195510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efffc00ead0 tx=0x7efffc00ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7efff6ffd700 1 -- 172.21.15.97:0/2357232950 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc013070 con 0x7f00000ff640 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7efff6ffd700 1 -- 172.21.15.97:0/2357232950 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc00be80 con 0x7f00000ff640 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0000067b20 con 0x7f00000ff640 2023-10-08T09:57:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.275+0000 7efff6ffd700 1 -- 172.21.15.97:0/2357232950 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0039e0 con 0x7f00000ff640 2023-10-08T09:57:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.276+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0000068030 con 0x7f00000ff640 2023-10-08T09:57:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.277+0000 7efff6ffd700 1 -- 172.21.15.97:0/2357232950 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efffc00b9c0 con 0x7f00000ff640 2023-10-08T09:57:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.278+0000 7efff6ffd700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 0x7effec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.278+0000 7efff6ffd700 1 -- 172.21.15.97:0/2357232950 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efffc0ce5d0 con 0x7f00000ff640 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.278+0000 7f00051db700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 0x7effec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.278+0000 7f00051db700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 0x7effec0a3940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efff0009fd0 tx=0x7efff000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 Processor -- start 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 -- start start 2023-10-08T09:57:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ee7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275ee7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275e4a400 con 0x558275ee76e0 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.279+0000 7f0013e29b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275e3bbc0 con 0x558275ee6200 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f0013e29b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275e394e0 con 0x558275ee6b60 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00061dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275ee7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37954/0 (socket says 172.21.15.97:37954) 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 -- 172.21.15.97:0/1252474165 learned_addr learned my addr 172.21.15.97:0/1252474165 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00061dd700 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275ee7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57988/0 (socket says 172.21.15.97:57988) 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00051db700 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ee7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 -- 172.21.15.97:0/1252474165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 msgr2=0x558275ee7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ee7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 -- 172.21.15.97:0/1252474165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 msgr2=0x558275ee7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275ee7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 -- 172.21.15.97:0/1252474165 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558275c5fd80 con 0x558275ee6200 2023-10-08T09:57:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7f00059dc700 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efffc0a3420 tx=0x7efffc09b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=226ff02bbc4255b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.280+0000 7effebfff700 1 -- 172.21.15.97:0/1252474165 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc09af20 con 0x558275ee6200 2023-10-08T09:57:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7effebfff700 1 -- 172.21.15.97:0/1252474165 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc09bcc0 con 0x558275ee6200 2023-10-08T09:57:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7effebfff700 1 -- 172.21.15.97:0/1252474165 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0ce370 con 0x558275ee6200 2023-10-08T09:57:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 -- 172.21.15.97:0/1252474165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 msgr2=0x558275ee6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efffc0a3420 tx=0x7efffc09b410 comp rx=0 tx=0).stop 2023-10-08T09:57:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 -- 172.21.15.97:0/1252474165 shutdown_connections 2023-10-08T09:57:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275ee7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6200 0x558275ee6620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1252474165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ee7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 -- 172.21.15.97:0/1252474165 >> 172.21.15.97:0/1252474165 conn(0x558275e56200 msgr2=0x558275edc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 -- 172.21.15.97:0/1252474165 shutdown_connections 2023-10-08T09:57:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.281+0000 7f0013e29b80 1 -- 172.21.15.97:0/1252474165 wait complete. 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 -- start start 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 Processor -- start 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275eefda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275e34dd0 con 0x558275ee6b60 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275dd16b0 con 0x558275ee76e0 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275dd1350 con 0x558275ee6200 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f0013e29b80 10 client.0 _ll_register_callbacks cb 0x558275c2e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.282+0000 7f00051db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00061dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57994/0 (socket says 172.21.15.97:57994) 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00061dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37962/0 (socket says 172.21.15.97:37962) 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 -- 172.21.15.97:0/1918762027 learned_addr learned my addr 172.21.15.97:0/1918762027 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00059dc700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275eefda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 -- 172.21.15.97:0/1918762027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 msgr2=0x558275eefda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275eefda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 -- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 msgr2=0x558275ef7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 -- 172.21.15.97:0/1918762027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0000068030 con 0x558275ee6b60 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00061dd700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f00051db700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efff00098e0 tx=0x7efff0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78b90b85969bcd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7effe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0029050 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efff0003e10 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558275dfce50 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff000f870 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f0013e29b80 10 client.16035 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.283+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558275c48c00 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558275dfce50 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efff0022a30 con 0x558275ee6b60 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 20 client.16035 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275efbc50 0x558275efe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 4 client.16035 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558275e2ddb0 con 0x558275efbc50 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.284+0000 7f00059dc700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275efbc50 0x558275efe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7effe97fa700 10 client.16035 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f00059dc700 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275efbc50 0x558275efe070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=808edc654415b39b server_cookie=e00db1e5c60194cc in_seq=0 out_seq=0 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7effe97fa700 1 -- 172.21.15.97:0/1918762027 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558275e2ddb0 con 0x558275efbc50 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7effe97fa700 10 client.16035 handle_command_reply: tid=0 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 client.16035 shutdown 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 msgr2=0x558275ef02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efff00098e0 tx=0x7efff0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 shutdown_connections 2023-10-08T09:57:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275efbc50 0x558275efe070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275eefda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 --2- 172.21.15.97:0/1918762027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6b60 0x558275ef02e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7effe97fa700 0 client.16035 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 >> 172.21.15.97:0/1918762027 conn(0x558275e56200 msgr2=0x558275edc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.285+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 shutdown_connections 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.286+0000 7f0013e29b80 1 -- 172.21.15.97:0/1918762027 wait complete. 2023-10-08T09:57:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.286+0000 7f0013e29b80 20 client.16035 trim_cache size 0 max 16384 2023-10-08T09:57:03.299 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/time.h usability... yes 2023-10-08T09:57:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.303+0000 7f0013e29b80 1 Processor -- start 2023-10-08T09:57:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 -- start start 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275ef7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 0x558275ef7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275eedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275cb58c0 con 0x558275ee76e0 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275dfce50 con 0x558275ee6b60 2023-10-08T09:57:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f0013e29b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275eef960 con 0x558275ee6200 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f00051db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 0x558275ef7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f00051db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 0x558275ef7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:37968/0 (socket says 172.21.15.97:37968) 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f00051db700 1 -- 172.21.15.97:0/2533087162 learned_addr learned my addr 172.21.15.97:0/2533087162 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.304+0000 7f00059dc700 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275ef7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00061dd700 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275eedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 -- 172.21.15.97:0/2533087162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 msgr2=0x558275ef7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 0x558275ef7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 -- 172.21.15.97:0/2533087162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 msgr2=0x558275eedd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275eedd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 -- 172.21.15.97:0/2533087162 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558275c861f0 con 0x558275ee6200 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.305+0000 7f00059dc700 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275ef7930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efffc0cbdf0 tx=0x7efffc09be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250579b380dcb1a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.306+0000 7efff4d82700 1 -- 172.21.15.97:0/2533087162 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0a3950 con 0x558275ee6200 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.306+0000 7efff4d82700 1 -- 172.21.15.97:0/2533087162 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc09af20 con 0x558275ee6200 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.306+0000 7efff4d82700 1 -- 172.21.15.97:0/2533087162 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc004120 con 0x558275ee6200 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.307+0000 7f0013e29b80 1 -- 172.21.15.97:0/2533087162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 msgr2=0x558275ef7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.307+0000 7f0013e29b80 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275ef7930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efffc0cbdf0 tx=0x7efffc09be30 comp rx=0 tx=0).stop 2023-10-08T09:57:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 -- 172.21.15.97:0/2533087162 shutdown_connections 2023-10-08T09:57:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee76e0 0x558275eedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6200 0x558275ef7930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 --2- 172.21.15.97:0/2533087162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee6b60 0x558275ef7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 -- 172.21.15.97:0/2533087162 >> 172.21.15.97:0/2533087162 conn(0x558275e56200 msgr2=0x558275f029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.308+0000 7f0013e29b80 1 -- 172.21.15.97:0/2533087162 shutdown_connections 2023-10-08T09:57:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.310+0000 7f0013e29b80 1 -- 172.21.15.97:0/2533087162 wait complete. 2023-10-08T09:57:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.310+0000 7f0013e29b80 1 -- start start 2023-10-08T09:57:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.310+0000 7f0013e29b80 1 Processor -- start 2023-10-08T09:57:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ef4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275ee5b90 con 0x558275ee6200 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275ee5d00 con 0x558275ee76e0 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558275ef6500 con 0x558275ee6b60 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f0013e29b80 10 client.0 _ll_register_callbacks cb 0x7efff0005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f00059dc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f00059dc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58004/0 (socket says 172.21.15.97:58004) 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f00059dc700 1 -- 172.21.15.97:0/265962551 learned_addr learned my addr 172.21.15.97:0/265962551 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f00051db700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ef4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.311+0000 7f00061dd700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 -- 172.21.15.97:0/265962551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 msgr2=0x558275ef4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ef4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 -- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 msgr2=0x558275ef5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 -- 172.21.15.97:0/265962551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558275c5fd80 con 0x558275ee6200 2023-10-08T09:57:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7f00059dc700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efffc0bb0f0 tx=0x7efffc0cbe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3640f82aedc67861 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7effe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0a3950 con 0x558275ee6200 2023-10-08T09:57:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.312+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efffc09b5c0 con 0x558275ee6200 2023-10-08T09:57:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efffc09bcc0 con 0x558275ee6200 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efffc0860d0 con 0x558275ee6200 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7f0013e29b80 10 client.16041 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558275c5e250 con 0x558275ee6200 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc09bcc0 con 0x558275ee6200 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.313+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efffc081070 con 0x558275ee6200 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 20 client.16041 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275f49be0 0x558275f4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 4 client.16041 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:57:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558275e2dad0 con 0x558275f49be0 2023-10-08T09:57:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f00051db700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275f49be0 0x558275f4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7effe97fa700 10 client.16041 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.314+0000 7f00051db700 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275f49be0 0x558275f4c000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a343d9c76a107d2e server_cookie=576e7bc44f46b495 in_seq=0 out_seq=0 2023-10-08T09:57:03.321 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/time.h presence... yes 2023-10-08T09:57:03.321 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/time.h... yes 2023-10-08T09:57:03.343 INFO:tasks.workunit.client.0.smithi097.stdout:checking for unistd.h... (cached) yes 2023-10-08T09:57:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7effe97fa700 1 -- 172.21.15.97:0/265962551 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558275e2dad0 con 0x558275f49be0 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7effe97fa700 10 client.16041 handle_command_reply: tid=0 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 client.16041 shutdown 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 msgr2=0x558275ef4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efffc0bb0f0 tx=0x7efffc0cbe20 comp rx=0 tx=0).stop 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 shutdown_connections 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558275ee76e0 0x558275ef5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558275ee6200 0x558275ef4a70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x558275f49be0 0x558275f4c000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 --2- 172.21.15.97:0/265962551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558275ee6b60 0x558275ef4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7effe97fa700 0 client.16041 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 >> 172.21.15.97:0/265962551 conn(0x558275e56200 msgr2=0x558275ed8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 shutdown_connections 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.432+0000 7f0013e29b80 1 -- 172.21.15.97:0/265962551 wait complete. 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0013e29b80 20 client.16041 trim_cache size 0 max 16384 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:57:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:57:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:57:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 msgr2=0x7effec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 0x7effec0a3940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efff0009fd0 tx=0x7efff000b540 comp rx=0 tx=0).stop 2023-10-08T09:57:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 msgr2=0x7f0000195510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f0000195510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efffc00ead0 tx=0x7efffc00ede0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 shutdown_connections 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7effec0a1480 0x7effec0a3940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00001011d0 0x7f000019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00000ff640 0x7f0000195510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 --2- 172.21.15.97:0/2357232950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0000100830 0x7f0000195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 >> 172.21.15.97:0/2357232950 conn(0x7f00000fabb0 msgr2=0x7f0000103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 shutdown_connections 2023-10-08T09:57:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:03.433+0000 7f0007c40700 1 -- 172.21.15.97:0/2357232950 wait complete. 2023-10-08T09:57:03.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: pgmap v357: 129 pgs: 129 active+clean; 3.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 217 KiB/s wr, 19 op/s 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub queued for path: / 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:03 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.457 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/vfs.h usability... yes 2023-10-08T09:57:03.470 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/vfs.h presence... yes 2023-10-08T09:57:03.470 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/vfs.h... yes 2023-10-08T09:57:03.519 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/limits.h usability... no 2023-10-08T09:57:03.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: pgmap v357: 129 pgs: 129 active+clean; 3.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 217 KiB/s wr, 19 op/s 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub queued for path: / 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:03 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.543 INFO:tasks.workunit.client.0.smithi097.stdout:checking sys/limits.h presence... no 2023-10-08T09:57:03.544 INFO:tasks.workunit.client.0.smithi097.stdout:checking for sys/limits.h... no 2023-10-08T09:57:03.569 INFO:tasks.workunit.client.0.smithi097.stdout:checking for an ANSI C-conforming const... yes 2023-10-08T09:57:03.594 INFO:tasks.workunit.client.0.smithi097.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-08T09:57:03.652 INFO:tasks.workunit.client.0.smithi097.stdout:checking for system... yes 2023-10-08T09:57:03.704 INFO:tasks.workunit.client.0.smithi097.stdout:checking for gettimeofday... yes 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: pgmap v357: 129 pgs: 129 active+clean; 3.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 217 KiB/s wr, 19 op/s 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub queued for path: / 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:03 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:57:03.756 INFO:tasks.workunit.client.0.smithi097.stdout:checking for mkdir... yes 2023-10-08T09:57:03.808 INFO:tasks.workunit.client.0.smithi097.stdout:checking for strerror... yes 2023-10-08T09:57:03.860 INFO:tasks.workunit.client.0.smithi097.stdout:checking for open64... yes 2023-10-08T09:57:03.913 INFO:tasks.workunit.client.0.smithi097.stdout:checking for stat64... yes 2023-10-08T09:57:03.965 INFO:tasks.workunit.client.0.smithi097.stdout:checking for fseeko64... yes 2023-10-08T09:57:04.018 INFO:tasks.workunit.client.0.smithi097.stdout:checking for lrand48_r... yes 2023-10-08T09:57:04.069 INFO:tasks.workunit.client.0.smithi097.stdout:checking for srand48_r... yes 2023-10-08T09:57:04.092 INFO:tasks.workunit.client.0.smithi097.stdout:configure: creating ./config.status 2023-10-08T09:57:04.151 INFO:tasks.workunit.client.0.smithi097.stdout: /bin/sh ./config.status 2023-10-08T09:57:04.196 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: creating Makefile 2023-10-08T09:57:04.207 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: creating config.h 2023-10-08T09:57:04.226 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: executing depfiles commands 2023-10-08T09:57:04.236 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: executing default commands 2023-10-08T09:57:04.260 INFO:tasks.workunit.client.0.smithi097.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2023-10-08T09:57:04.395 INFO:tasks.workunit.client.0.smithi097.stdout:rm -f stamp-h1 2023-10-08T09:57:04.398 INFO:tasks.workunit.client.0.smithi097.stdout:touch config.h.in 2023-10-08T09:57:04.399 INFO:tasks.workunit.client.0.smithi097.stdout:cd . && /bin/sh ./config.status config.h 2023-10-08T09:57:04.443 INFO:tasks.workunit.client.0.smithi097.stdout:config.status: creating config.h 2023-10-08T09:57:04.468 INFO:tasks.workunit.client.0.smithi097.stdout:make all-am 2023-10-08T09:57:04.474 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-10-08T09:57:04.474 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2023-10-08T09:57:04.615 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2023-10-08T09:57:04.618 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2023-10-08T09:57:04.681 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2023-10-08T09:57:04.683 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2023-10-08T09:57:04.768 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/main.Tpo .deps/main.Po 2023-10-08T09:57:04.771 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2023-10-08T09:57:04.858 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2023-10-08T09:57:04.861 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2023-10-08T09:57:04.965 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2023-10-08T09:57:04.968 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2023-10-08T09:57:05.015 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2023-10-08T09:57:05.018 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2023-10-08T09:57:05.056 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2023-10-08T09:57:05.058 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2023-10-08T09:57:05.101 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2023-10-08T09:57:05.104 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2023-10-08T09:57:05.256 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2023-10-08T09:57:05.260 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2023-10-08T09:57:05.367 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2023-10-08T09:57:05.370 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2023-10-08T09:57:05.501 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2023-10-08T09:57:05.504 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2023-10-08T09:57:05.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:05 smithi186 ceph-mon[104393]: pgmap v358: 129 pgs: 129 active+clean; 3.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-10-08T09:57:05.566 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2023-10-08T09:57:05.569 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2023-10-08T09:57:05.632 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2023-10-08T09:57:05.636 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2023-10-08T09:57:05.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:05 smithi097 ceph-mon[99491]: pgmap v358: 129 pgs: 129 active+clean; 3.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-10-08T09:57:05.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:05 smithi178 ceph-mon[103156]: pgmap v358: 129 pgs: 129 active+clean; 3.3 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-10-08T09:57:05.730 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/util.Tpo .deps/util.Po 2023-10-08T09:57:05.733 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2023-10-08T09:57:05.759 INFO:tasks.workunit.client.0.smithi097.stderr:In file included from parser.h:22, 2023-10-08T09:57:05.759 INFO:tasks.workunit.client.0.smithi097.stderr: from parser.c:25: 2023-10-08T09:57:05.759 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c: In function ‘init_filesys’: 2023-10-08T09:57:05.759 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.760 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.760 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.760 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.760 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.761 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.761 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.761 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.761 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.761 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.762 INFO:tasks.workunit.client.0.smithi097.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-08T09:57:05.762 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.762 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.762 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-10-08T09:57:05.762 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.763 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.763 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.763 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.763 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.763 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.764 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.765 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.766 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.767 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.768 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c: In function ‘init_tg_stats’: 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.769 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.770 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.771 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.772 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.773 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-08T09:57:05.774 INFO:tasks.workunit.client.0.smithi097.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr: ^ 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~ 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr: container_of(ptr, type, member) 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~ 2023-10-08T09:57:05.775 INFO:tasks.workunit.client.0.smithi097.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-08T09:57:05.776 INFO:tasks.workunit.client.0.smithi097.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-08T09:57:05.776 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~ 2023-10-08T09:57:05.776 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-08T09:57:05.776 INFO:tasks.workunit.client.0.smithi097.stderr: list_for_each_entry(tmp_list, 2023-10-08T09:57:05.776 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c: In function ‘set_weight’: 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:623:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr: strncpy (op, tmp_config->name, len - 7); 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:620:10: note: length computed here 2023-10-08T09:57:05.863 INFO:tasks.workunit.client.0.smithi097.stderr: len = strlen(tmp_config->name); 2023-10-08T09:57:05.864 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.867 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c: In function ‘get_weight_total’: 2023-10-08T09:57:05.867 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:645:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-10-08T09:57:05.867 INFO:tasks.workunit.client.0.smithi097.stderr: strncpy (op, tmp_config->name, len - 7); 2023-10-08T09:57:05.868 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.868 INFO:tasks.workunit.client.0.smithi097.stderr:parser.c:642:10: note: length computed here 2023-10-08T09:57:05.868 INFO:tasks.workunit.client.0.smithi097.stderr: len = strlen(tmp_config->name); 2023-10-08T09:57:05.868 INFO:tasks.workunit.client.0.smithi097.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-08T09:57:05.989 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2023-10-08T09:57:05.998 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2023-10-08T09:57:06.045 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2023-10-08T09:57:06.053 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2023-10-08T09:57:06.131 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2023-10-08T09:57:06.151 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2023-10-08T09:57:06.163 INFO:tasks.workunit.client.0.smithi097.stdout:mv -f .deps/list.Tpo .deps/list.Po 2023-10-08T09:57:06.179 INFO:tasks.workunit.client.0.smithi097.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2023-10-08T09:57:06.197 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-10-08T09:57:06.208 INFO:tasks.workunit.client.0.smithi097.stderr:+ cd .. 2023-10-08T09:57:06.209 INFO:tasks.workunit.client.0.smithi097.stderr:+ mkdir tmp 2023-10-08T09:57:06.210 INFO:tasks.workunit.client.0.smithi097.stderr:+ cd tmp 2023-10-08T09:57:06.210 INFO:tasks.workunit.client.0.smithi097.stderr:+ for f in $mydir/*.ffsb 2023-10-08T09:57:06.211 INFO:tasks.workunit.client.0.smithi097.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2023-10-08T09:57:06.211 INFO:tasks.workunit.client.0.smithi097.stdout:FFSB version 6.0-RC2 started 2023-10-08T09:57:06.211 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.213 INFO:tasks.workunit.client.0.smithi097.stdout:benchmark time = 300 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout:ThreadGroup 0 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout:================ 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: num_threads = 32 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: read_random = off 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: read_size = 0 (0B) 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: read_blocksize = 0 (0B) 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: read_skip = off 2023-10-08T09:57:06.214 INFO:tasks.workunit.client.0.smithi097.stdout: read_skipsize = 0 (0B) 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: write_random = on 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: write_size = 5242880 (5MB) 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: fsync_file = 0 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: write_blocksize = 4096 (4KB) 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: wait time = 0 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: op weights 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: read = 0 (0.00%) 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: readall = 0 (0.00%) 2023-10-08T09:57:06.215 INFO:tasks.workunit.client.0.smithi097.stdout: write = 1 (100.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: create = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: append = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: delete = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: metaop = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: createdir = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: stat = 0 (0.00%) 2023-10-08T09:57:06.216 INFO:tasks.workunit.client.0.smithi097.stdout: writeall = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: writeall_fsync = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: open_close = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: write_fsync = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: create_fsync = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: append_fsync = 0 (0.00%) 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout:FileSystem . 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout:========== 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: num_dirs = 0 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: starting files = 128 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.217 INFO:tasks.workunit.client.0.smithi097.stdout: min file size = 104857600 (100MB) 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: max file size = 104857600 (100MB) 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: directio = off 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: alignedio = on 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: bufferedio = off 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: aging is off 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: current utilization = 0.03% 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:06.218 INFO:tasks.workunit.client.0.smithi097.stdout:checking existing fs: . 2023-10-08T09:57:06.219 INFO:tasks.workunit.client.0.smithi097.stdout:recreating new fileset 2023-10-08T09:57:06.219 INFO:tasks.workunit.client.0.smithi097.stderr:opendir: No such file or directory 2023-10-08T09:57:06.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:06 smithi097 ceph-mon[99491]: pgmap v359: 129 pgs: 129 active+clean; 4.7 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 426 KiB/s wr, 101 op/s 2023-10-08T09:57:06.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:06 smithi178 ceph-mon[103156]: pgmap v359: 129 pgs: 129 active+clean; 4.7 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 426 KiB/s wr, 101 op/s 2023-10-08T09:57:06.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:06 smithi186 ceph-mon[104393]: pgmap v359: 129 pgs: 129 active+clean; 4.7 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 426 KiB/s wr, 101 op/s 2023-10-08T09:57:09.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:09 smithi178 ceph-mon[103156]: pgmap v360: 129 pgs: 129 active+clean; 50 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 4.2 MiB/s wr, 133 op/s 2023-10-08T09:57:09.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:09 smithi097 ceph-mon[99491]: pgmap v360: 129 pgs: 129 active+clean; 50 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 4.2 MiB/s wr, 133 op/s 2023-10-08T09:57:09.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:09 smithi186 ceph-mon[104393]: pgmap v360: 129 pgs: 129 active+clean; 50 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 4.2 MiB/s wr, 133 op/s 2023-10-08T09:57:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:11 smithi097 ceph-mon[99491]: pgmap v361: 129 pgs: 129 active+clean; 202 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 17 MiB/s wr, 142 op/s 2023-10-08T09:57:11.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:11 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T09:57:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:11 smithi178 ceph-mon[103156]: pgmap v361: 129 pgs: 129 active+clean; 202 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 17 MiB/s wr, 142 op/s 2023-10-08T09:57:11.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:11 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T09:57:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:11 smithi186 ceph-mon[104393]: pgmap v361: 129 pgs: 129 active+clean; 202 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 17 MiB/s wr, 142 op/s 2023-10-08T09:57:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:11 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T09:57:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:12 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T09:57:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:12 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T09:57:12.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:12 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:57:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:12 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T09:57:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:12 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T09:57:12.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:12 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:57:12.712 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:12 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T09:57:12.712 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:12 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T09:57:12.712 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:12 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:57:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:13 smithi178 ceph-mon[103156]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 119 MiB/s wr, 232 op/s 2023-10-08T09:57:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:57:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:57:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:57:13.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:13 smithi097 ceph-mon[99491]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 119 MiB/s wr, 232 op/s 2023-10-08T09:57:13.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:57:13.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:57:13.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:57:13.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:13 smithi186 ceph-mon[104393]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 119 MiB/s wr, 232 op/s 2023-10-08T09:57:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:57:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:57:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:57:14.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:14 smithi097 ceph-mon[99491]: pgmap v363: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 MiB/s wr, 256 op/s 2023-10-08T09:57:14.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:14 smithi178 ceph-mon[103156]: pgmap v363: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 MiB/s wr, 256 op/s 2023-10-08T09:57:14.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:14 smithi186 ceph-mon[104393]: pgmap v363: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 MiB/s wr, 256 op/s 2023-10-08T09:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 -- 172.21.15.97:0/314773633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b481077f0 msgr2=0x7f0b48109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 --2- 172.21.15.97:0/314773633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b481077f0 0x7f0b48109c20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c01a9a0 tx=0x7f0b3c01acb0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 -- 172.21.15.97:0/314773633 shutdown_connections 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 --2- 172.21.15.97:0/314773633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 --2- 172.21.15.97:0/314773633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b481077f0 0x7f0b48109c20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 --2- 172.21.15.97:0/314773633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b4810a160 0x7f0b4810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.771+0000 7f0b4e780700 1 -- 172.21.15.97:0/314773633 >> 172.21.15.97:0/314773633 conn(0x7f0b4806d770 msgr2=0x7f0b4806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.772+0000 7f0b4e780700 1 -- 172.21.15.97:0/314773633 shutdown_connections 2023-10-08T09:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.772+0000 7f0b4e780700 1 -- 172.21.15.97:0/314773633 wait complete. 2023-10-08T09:57:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.773+0000 7f0b4e780700 1 Processor -- start 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.773+0000 7f0b4e780700 1 -- start start 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b481a2460 0x7f0b481a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481a4e20 con 0x7f0b4810a160 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481a4f60 con 0x7f0b4810cb30 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4e780700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481a50f0 con 0x7f0b481a2460 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58018/0 (socket says 172.21.15.97:58018) 2023-10-08T09:57:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b47fff700 1 -- 172.21.15.97:0/3171096350 learned_addr learned my addr 172.21.15.97:0/3171096350 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.774+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b481a2460 0x7f0b481a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 -- 172.21.15.97:0/3171096350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b481a2460 msgr2=0x7f0b481a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b481a2460 0x7f0b481a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 -- 172.21.15.97:0/3171096350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 msgr2=0x7f0b4819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 -- 172.21.15.97:0/3171096350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c01a600 con 0x7f0b4810a160 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b457fa700 1 -- 172.21.15.97:0/3171096350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00d660 con 0x7f0b4810a160 2023-10-08T09:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b481a5370 con 0x7f0b4810a160 2023-10-08T09:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b457fa700 1 -- 172.21.15.97:0/3171096350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3c01a040 con 0x7f0b4810a160 2023-10-08T09:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.775+0000 7f0b457fa700 1 -- 172.21.15.97:0/3171096350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00c620 con 0x7f0b4810a160 2023-10-08T09:57:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.776+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b481a56e0 con 0x7f0b4810a160 2023-10-08T09:57:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.780+0000 7f0b457fa700 1 -- 172.21.15.97:0/3171096350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0b3c00c780 con 0x7f0b4810a160 2023-10-08T09:57:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.780+0000 7f0b5a969b80 1 Processor -- start 2023-10-08T09:57:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.780+0000 7f0b5a969b80 1 -- start start 2023-10-08T09:57:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da792d400 con 0x559da79c4e40 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da791ebc0 con 0x559da79c62b0 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da791c4e0 con 0x559da79c57a0 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b457fa700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 0x7f0b300c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b4cd1d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b4cd1d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58024/0 (socket says 172.21.15.97:58024) 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b4cd1d700 1 -- 172.21.15.97:0/2523943708 learned_addr learned my addr 172.21.15.97:0/2523943708 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 0x7f0b300c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b457fa700 1 -- 172.21.15.97:0/3171096350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0b3c0d0220 con 0x7f0b4810a160 2023-10-08T09:57:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.781+0000 7f0b47fff700 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 -- 172.21.15.97:0/2523943708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 msgr2=0x559da79c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 -- 172.21.15.97:0/2523943708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 msgr2=0x559da79c6730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79c6730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 -- 172.21.15.97:0/2523943708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da7742d80 con 0x559da79c4e40 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0b40007ed0 tx=0x7f0b40018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b4cf626e65bc00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.782+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 0x7f0b300c5950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b40018a40 tx=0x7f0b4000b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.784+0000 7f0b2e7fc700 1 -- 172.21.15.97:0/2523943708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000e040 con 0x559da79c4e40 2023-10-08T09:57:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.784+0000 7f0b2e7fc700 1 -- 172.21.15.97:0/2523943708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4001acc0 con 0x559da79c4e40 2023-10-08T09:57:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.784+0000 7f0b2e7fc700 1 -- 172.21.15.97:0/2523943708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40005280 con 0x559da79c4e40 2023-10-08T09:57:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- 172.21.15.97:0/2523943708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 msgr2=0x559da79c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0b40007ed0 tx=0x7f0b40018640 comp rx=0 tx=0).stop 2023-10-08T09:57:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- 172.21.15.97:0/2523943708 shutdown_connections 2023-10-08T09:57:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c5260 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/2523943708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- 172.21.15.97:0/2523943708 >> 172.21.15.97:0/2523943708 conn(0x559da7939200 msgr2=0x559da79bf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- 172.21.15.97:0/2523943708 shutdown_connections 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- 172.21.15.97:0/2523943708 wait complete. 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.785+0000 7f0b5a969b80 1 -- start start 2023-10-08T09:57:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 Processor -- start 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c57a0 0x559da79cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c62b0 0x559da79d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da7917dd0 con 0x559da79c4e40 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da78b46b0 con 0x559da79c57a0 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da78b4350 con 0x559da79c62b0 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b5a969b80 10 client.0 _ll_register_callbacks cb 0x559da77117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58030/0 (socket says 172.21.15.97:58030) 2023-10-08T09:57:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b47fff700 1 -- 172.21.15.97:0/3571363185 learned_addr learned my addr 172.21.15.97:0/3571363185 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.786+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c57a0 0x559da79cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 -- 172.21.15.97:0/3571363185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c62b0 msgr2=0x559da79d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c62b0 0x559da79d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 -- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c57a0 msgr2=0x559da79cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c57a0 0x559da79cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 -- 172.21.15.97:0/3571363185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b40007ab0 con 0x559da79c4e40 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c097620 tx=0x7f0b3c0974e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c841273039e641c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b277fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c09aa80 con 0x559da79c4e40 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da78dfe50 con 0x559da79c4e40 2023-10-08T09:57:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.787+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0b3c00b320 con 0x559da79c4e40 2023-10-08T09:57:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b3c09b820 con 0x559da79c4e40 2023-10-08T09:57:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 10 client.16053 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559da772bc00 con 0x559da79c4e40 2023-10-08T09:57:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559da78dfe50 con 0x559da79c4e40 2023-10-08T09:57:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0b3c09b380 con 0x559da79c4e40 2023-10-08T09:57:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 20 client.16053 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da79da190 0x559da79dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 4 client.16053 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.788+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da7910db0 con 0x559da79da190 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.789+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da79da190 0x559da79dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.790+0000 7f0b277fe700 10 client.16053 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.790+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da79da190 0x559da79dc5b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cd8cdea484455d server_cookie=4c61c1d41a73dee in_seq=0 out_seq=0 2023-10-08T09:57:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.791+0000 7f0b277fe700 1 -- 172.21.15.97:0/3571363185 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559da7910db0 con 0x559da79da190 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.791+0000 7f0b277fe700 10 client.16053 handle_command_reply: tid=0 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.791+0000 7f0b5a969b80 1 client.16053 shutdown 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.791+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 msgr2=0x559da79ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.791+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c097620 tx=0x7f0b3c0974e0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 shutdown_connections 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da79da190 0x559da79dc5b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c62b0 0x559da79d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79ce8b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3571363185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c57a0 0x559da79cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 >> 172.21.15.97:0/3571363185 conn(0x559da7939200 msgr2=0x559da79bf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b277fe700 0 client.16053 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 shutdown_connections 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3571363185 wait complete. 2023-10-08T09:57:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.792+0000 7f0b5a969b80 20 client.16053 trim_cache size 0 max 16384 2023-10-08T09:57:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 Processor -- start 2023-10-08T09:57:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 -- start start 2023-10-08T09:57:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da77988c0 con 0x559da79c4e40 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da79cd2f0 con 0x559da79c62b0 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da79cc530 con 0x559da79c57a0 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58038/0 (socket says 172.21.15.97:58038) 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.817+0000 7f0b47fff700 1 -- 172.21.15.97:0/3791911769 learned_addr learned my addr 172.21.15.97:0/3791911769 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 -- 172.21.15.97:0/3791911769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 msgr2=0x559da79d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 -- 172.21.15.97:0/3791911769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 msgr2=0x559da79cc8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79cc8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 -- 172.21.15.97:0/3791911769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da77691f0 con 0x559da79c4e40 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c003790 tx=0x7f0b3c0996d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc56cb92db46e677 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b2effd700 1 -- 172.21.15.97:0/3791911769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c09aa80 con 0x559da79c4e40 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b2effd700 1 -- 172.21.15.97:0/3791911769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3c09aef0 con 0x559da79c4e40 2023-10-08T09:57:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.818+0000 7f0b2effd700 1 -- 172.21.15.97:0/3791911769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c09f270 con 0x559da79c4e40 2023-10-08T09:57:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3791911769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 msgr2=0x559da79c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c003790 tx=0x7f0b3c0996d0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3791911769 shutdown_connections 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79c6690 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3791911769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3791911769 >> 172.21.15.97:0/3791911769 conn(0x559da7939200 msgr2=0x559da79e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3791911769 shutdown_connections 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.819+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3791911769 wait complete. 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.820+0000 7f0b5a969b80 1 -- start start 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.820+0000 7f0b5a969b80 1 Processor -- start 2023-10-08T09:57:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da79d5040 con 0x559da79c4e40 2023-10-08T09:57:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da79d51b0 con 0x559da79c62b0 2023-10-08T09:57:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da79d5370 con 0x559da79c57a0 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b5a969b80 10 client.0 _ll_register_callbacks cb 0x559da77117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58044/0 (socket says 172.21.15.97:58044) 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.821+0000 7f0b47fff700 1 -- 172.21.15.97:0/3567284199 learned_addr learned my addr 172.21.15.97:0/3567284199 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b4cd1d700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 -- 172.21.15.97:0/3567284199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 msgr2=0x559da79d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 -- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 msgr2=0x559da79d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 -- 172.21.15.97:0/3567284199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0b3070 con 0x559da79c4e40 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b47fff700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0a2ef0 tx=0x7f0b3c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8026c05edfe1c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b277fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.822+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c09b190 con 0x559da79c4e40 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0b3c0a3420 con 0x559da79c4e40 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b4001acc0 con 0x559da79c4e40 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b3c09aa80 con 0x559da79c4e40 2023-10-08T09:57:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 10 client.16059 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559da7741250 con 0x559da79c4e40 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4001acc0 con 0x559da79c4e40 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0b3c066020 con 0x559da79c4e40 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.823+0000 7f0b5a969b80 20 client.16059 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.824+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da7a2e4a0 0x559da7a30890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.824+0000 7f0b5a969b80 4 client.16059 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:57:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.824+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559da7910ad0 con 0x559da7a2e4a0 2023-10-08T09:57:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.824+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da7a2e4a0 0x559da7a30890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.826+0000 7f0b277fe700 10 client.16059 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:16.826+0000 7f0b477fe700 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da7a2e4a0 0x559da7a30890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c65392a818676c server_cookie=84e3f71511577eed in_seq=0 out_seq=0 2023-10-08T09:57:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:17 smithi186 ceph-mon[104393]: pgmap v364: 129 pgs: 129 active+clean; 5.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 449 MiB/s wr, 335 op/s 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b277fe700 1 -- 172.21.15.97:0/3567284199 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559da7910ad0 con 0x559da7a2e4a0 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b277fe700 10 client.16059 handle_command_reply: tid=0 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 client.16059 shutdown 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 msgr2=0x559da79d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0a2ef0 tx=0x7f0b3c01acb0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 shutdown_connections 2023-10-08T09:57:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559da79c62b0 0x559da79d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559da79c4e40 0x559da79d35b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.639+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x559da7a2e4a0 0x559da7a30890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b5a969b80 1 --2- 172.21.15.97:0/3567284199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559da79c57a0 0x559da79d3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 >> 172.21.15.97:0/3567284199 conn(0x559da7939200 msgr2=0x559da79cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b277fe700 0 client.16059 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 shutdown_connections 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b5a969b80 1 -- 172.21.15.97:0/3567284199 wait complete. 2023-10-08T09:57:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b5a969b80 20 client.16059 trim_cache size 0 max 16384 2023-10-08T09:57:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:57:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:57:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:57:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 msgr2=0x7f0b300c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 0x7f0b300c5950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b40018a40 tx=0x7f0b4000b000 comp rx=0 tx=0).stop 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 msgr2=0x7f0b4819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.640+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c003f80 comp rx=0 tx=0).stop 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 shutdown_connections 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0b300c3490 0x7f0b300c5950 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b481a2460 0x7f0b481a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b4810a160 0x7f0b4819cf10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 --2- 172.21.15.97:0/3171096350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4810cb30 0x7f0b4819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 >> 172.21.15.97:0/3171096350 conn(0x7f0b4806d770 msgr2=0x7f0b48108110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 shutdown_connections 2023-10-08T09:57:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:17.641+0000 7f0b4e780700 1 -- 172.21.15.97:0/3171096350 wait complete. 2023-10-08T09:57:17.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:17 smithi097 ceph-mon[99491]: pgmap v364: 129 pgs: 129 active+clean; 5.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 449 MiB/s wr, 335 op/s 2023-10-08T09:57:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:17 smithi178 ceph-mon[103156]: pgmap v364: 129 pgs: 129 active+clean; 5.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 449 MiB/s wr, 335 op/s 2023-10-08T09:57:19.617 INFO:tasks.workunit.client.0.smithi097.stdout:fs setup took 13 secs 2023-10-08T09:57:19.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:19 smithi178 ceph-mon[103156]: pgmap v365: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 554 MiB/s wr, 296 op/s 2023-10-08T09:57:19.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:19 smithi097 ceph-mon[99491]: pgmap v365: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 554 MiB/s wr, 296 op/s 2023-10-08T09:57:19.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:19 smithi186 ceph-mon[104393]: pgmap v365: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 554 MiB/s wr, 296 op/s 2023-10-08T09:57:21.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:21 smithi186 ceph-mon[104393]: pgmap v366: 129 pgs: 129 active+clean; 7.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 610 MiB/s wr, 279 op/s 2023-10-08T09:57:21.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:21 smithi097 ceph-mon[99491]: pgmap v366: 129 pgs: 129 active+clean; 7.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 610 MiB/s wr, 279 op/s 2023-10-08T09:57:21.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:21 smithi178 ceph-mon[103156]: pgmap v366: 129 pgs: 129 active+clean; 7.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 610 MiB/s wr, 279 op/s 2023-10-08T09:57:22.346 INFO:tasks.workunit.client.0.smithi097.stdout:Syncing()...2 sec 2023-10-08T09:57:22.346 INFO:tasks.workunit.client.0.smithi097.stdout:Starting Actual Benchmark At: Sun Oct 8 09:57:22 2023 2023-10-08T09:57:22.346 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T09:57:23.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:23 smithi186 ceph-mon[104393]: pgmap v367: 129 pgs: 129 active+clean; 9.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 810 MiB/s wr, 364 op/s 2023-10-08T09:57:23.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:23 smithi097 ceph-mon[99491]: pgmap v367: 129 pgs: 129 active+clean; 9.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 810 MiB/s wr, 364 op/s 2023-10-08T09:57:23.711 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:23 smithi178 ceph-mon[103156]: pgmap v367: 129 pgs: 129 active+clean; 9.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 810 MiB/s wr, 364 op/s 2023-10-08T09:57:25.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:25 smithi186 ceph-mon[104393]: pgmap v368: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 814 MiB/s wr, 301 op/s 2023-10-08T09:57:25.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:25 smithi178 ceph-mon[103156]: pgmap v368: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 814 MiB/s wr, 301 op/s 2023-10-08T09:57:25.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:25 smithi097 ceph-mon[99491]: pgmap v368: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 814 MiB/s wr, 301 op/s 2023-10-08T09:57:27.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:27 smithi097 ceph-mon[99491]: pgmap v369: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 827 MiB/s wr, 1.64k op/s 2023-10-08T09:57:27.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:27 smithi178 ceph-mon[103156]: pgmap v369: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 827 MiB/s wr, 1.64k op/s 2023-10-08T09:57:27.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:27 smithi186 ceph-mon[104393]: pgmap v369: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 827 MiB/s wr, 1.64k op/s 2023-10-08T09:57:29.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:29 smithi097 ceph-mon[99491]: pgmap v370: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 640 MiB/s wr, 4.03k op/s 2023-10-08T09:57:29.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:29 smithi178 ceph-mon[103156]: pgmap v370: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 640 MiB/s wr, 4.03k op/s 2023-10-08T09:57:29.781 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:29 smithi186 ceph-mon[104393]: pgmap v370: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 640 MiB/s wr, 4.03k op/s 2023-10-08T09:57:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.859+0000 7f76de655700 1 -- 172.21.15.97:0/3360097053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d81077f0 msgr2=0x7f76d8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.859+0000 7f76de655700 1 --2- 172.21.15.97:0/3360097053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d81077f0 0x7f76d8109c20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f76c80098f0 tx=0x7f76c8009c00 comp rx=0 tx=0).stop 2023-10-08T09:57:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.866+0000 7f76de655700 1 -- 172.21.15.97:0/3360097053 shutdown_connections 2023-10-08T09:57:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.866+0000 7f76de655700 1 --2- 172.21.15.97:0/3360097053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810cb30 0x7f76d810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.866+0000 7f76de655700 1 --2- 172.21.15.97:0/3360097053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d81077f0 0x7f76d8109c20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.866+0000 7f76de655700 1 --2- 172.21.15.97:0/3360097053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d810a160 0x7f76d810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.866+0000 7f76de655700 1 -- 172.21.15.97:0/3360097053 >> 172.21.15.97:0/3360097053 conn(0x7f76d806d770 msgr2=0x7f76d806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.877+0000 7f76de655700 1 -- 172.21.15.97:0/3360097053 shutdown_connections 2023-10-08T09:57:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.878+0000 7f76de655700 1 -- 172.21.15.97:0/3360097053 wait complete. 2023-10-08T09:57:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 Processor -- start 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 -- start start 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810a160 0x7f76d819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d810cb30 0x7f76d8078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8079050 con 0x7f76d810cb30 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80791c0 con 0x7f76d810a160 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.879+0000 7f76de655700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8079330 con 0x7f76d81077f0 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.880+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.880+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59758/0 (socket says 172.21.15.97:59758) 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.880+0000 7f76d7fff700 1 -- 172.21.15.97:0/106105855 learned_addr learned my addr 172.21.15.97:0/106105855 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.880+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d810cb30 0x7f76d8078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.884+0000 7f76d77fe700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810a160 0x7f76d819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 -- 172.21.15.97:0/106105855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810a160 msgr2=0x7f76d819d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810a160 0x7f76d819d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 -- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d810cb30 msgr2=0x7f76d8078a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d810cb30 0x7f76d8078a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 -- 172.21.15.97:0/106105855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8009520 con 0x7f76d81077f0 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.891+0000 7f76d7fff700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c010 tx=0x7f76c800bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.895+0000 7f76d57fa700 1 -- 172.21.15.97:0/106105855 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8005520 con 0x7f76d81077f0 2023-10-08T09:57:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.895+0000 7f76d57fa700 1 -- 172.21.15.97:0/106105855 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c8005680 con 0x7f76d81077f0 2023-10-08T09:57:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.895+0000 7f76d57fa700 1 -- 172.21.15.97:0/106105855 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80178b0 con 0x7f76d81077f0 2023-10-08T09:57:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.896+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80795b0 con 0x7f76d81077f0 2023-10-08T09:57:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.896+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d80799e0 con 0x7f76d81077f0 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 Processor -- start 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 -- start start 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e947a0 0x55b1e0e94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0dfc400 con 0x55b1e0e93e40 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0dedbc0 con 0x55b1e0e952b0 2023-10-08T09:57:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.906+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0deb4e0 con 0x55b1e0e947a0 2023-10-08T09:57:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.913+0000 7f76dcbf2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.913+0000 7f76dcbf2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58064/0 (socket says 172.21.15.97:58064) 2023-10-08T09:57:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.913+0000 7f76dcbf2700 1 -- 172.21.15.97:0/1928738365 learned_addr learned my addr 172.21.15.97:0/1928738365 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.913+0000 7f76d57fa700 1 -- 172.21.15.97:0/106105855 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f76c8003bd0 con 0x7f76d81077f0 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.914+0000 7f76d57fa700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 0x7f76c00a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.914+0000 7f76d57fa700 1 -- 172.21.15.97:0/106105855 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f76c80cde90 con 0x7f76d81077f0 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.915+0000 7f76d77fe700 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.915+0000 7f76d7fff700 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e947a0 0x55b1e0e94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 -- 172.21.15.97:0/1928738365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e947a0 msgr2=0x55b1e0e94cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e947a0 0x55b1e0e94cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 -- 172.21.15.97:0/1928738365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 msgr2=0x55b1e0e95730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e95730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 -- 172.21.15.97:0/1928738365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e0c11d80 con 0x55b1e0e93e40 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 0x7f76c00a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009f60 tx=0x7f76d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4126f76345dc750b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76be7fc700 1 -- 172.21.15.97:0/1928738365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0018db0 con 0x55b1e0e93e40 2023-10-08T09:57:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.917+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 0x7f76c00a3b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76d0020000 tx=0x7f76d001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.918+0000 7f76be7fc700 1 -- 172.21.15.97:0/1928738365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d000f050 con 0x55b1e0e93e40 2023-10-08T09:57:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.918+0000 7f76be7fc700 1 -- 172.21.15.97:0/1928738365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00175d0 con 0x55b1e0e93e40 2023-10-08T09:57:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.918+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1928738365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 msgr2=0x55b1e0e94260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.918+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009f60 tx=0x7f76d0009f90 comp rx=0 tx=0).stop 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.919+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1928738365 shutdown_connections 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.919+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.919+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94260 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.919+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1928738365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e947a0 0x55b1e0e94cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.919+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1928738365 >> 172.21.15.97:0/1928738365 conn(0x55b1e0e08200 msgr2=0x55b1e0e8e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.920+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1928738365 shutdown_connections 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.920+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1928738365 wait complete. 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.920+0000 7f76ea83eb80 1 -- start start 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 Processor -- start 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0e9de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0de6dd0 con 0x55b1e0e947a0 2023-10-08T09:57:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0d836b0 con 0x55b1e0e952b0 2023-10-08T09:57:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0d83350 con 0x55b1e0e93e40 2023-10-08T09:57:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76ea83eb80 10 client.0 _ll_register_callbacks cb 0x55b1e0be07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59774/0 (socket says 172.21.15.97:59774) 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76d7fff700 1 -- 172.21.15.97:0/180907716 learned_addr learned my addr 172.21.15.97:0/180907716 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.921+0000 7f76d77fe700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0e9de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76dcbf2700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76d7fff700 1 -- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 msgr2=0x55b1e0ea4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76d7fff700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76d7fff700 1 -- 172.21.15.97:0/180907716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 msgr2=0x55b1e0e9de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76d7fff700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0e9de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.922+0000 7f76d7fff700 1 -- 172.21.15.97:0/180907716 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d0009a70 con 0x55b1e0e93e40 2023-10-08T09:57:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.923+0000 7f76d7fff700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76c80973b0 tx=0x7f76c809a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4ecaf01ea3930a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.928+0000 7f76b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:57:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.928+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c809f000 con 0x55b1e0e93e40 2023-10-08T09:57:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.928+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f76c80a31b0 con 0x55b1e0e93e40 2023-10-08T09:57:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.928+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e0daee50 con 0x55b1e0e93e40 2023-10-08T09:57:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.932+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c80cb710 con 0x55b1e0e93e40 2023-10-08T09:57:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.935+0000 7f76ea83eb80 10 client.25462 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.935+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b1e0bfac00 con 0x55b1e0e93e40 2023-10-08T09:57:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1e0daee50 con 0x55b1e0e93e40 2023-10-08T09:57:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f76c8003680 con 0x55b1e0e93e40 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 10 client.25462 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 20 client.25462 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0ecb3b0 0x55b1e0ecd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 4 client.25462 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.937+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1e0ddfdb0 con 0x55b1e0ecb3b0 2023-10-08T09:57:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.938+0000 7f76d77fe700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0ecb3b0 0x55b1e0ecd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.955+0000 7f76b77fe700 10 client.25462 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.955+0000 7f76d77fe700 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0ecb3b0 0x55b1e0ecd7d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b25e2fe1c43e5b server_cookie=c62e14705dca9425 in_seq=0 out_seq=0 2023-10-08T09:57:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.961+0000 7f76b77fe700 1 -- 172.21.15.97:0/180907716 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b1e0ddfdb0 con 0x55b1e0ecb3b0 2023-10-08T09:57:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.961+0000 7f76b77fe700 10 client.25462 handle_command_reply: tid=0 2023-10-08T09:57:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.967+0000 7f76ea83eb80 1 client.25462 shutdown 2023-10-08T09:57:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.975+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 msgr2=0x55b1e0e9d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.975+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76c80973b0 tx=0x7f76c809a8c0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 shutdown_connections 2023-10-08T09:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0ecb3b0 0x55b1e0ecd7d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e9d8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/180907716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0e9de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.979+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 >> 172.21.15.97:0/180907716 conn(0x55b1e0e08200 msgr2=0x55b1e0e8e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:30.984+0000 7f76b77fe700 0 client.25462 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.004+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 shutdown_connections 2023-10-08T09:57:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.012+0000 7f76ea83eb80 1 -- 172.21.15.97:0/180907716 wait complete. 2023-10-08T09:57:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.014+0000 7f76ea83eb80 20 client.25462 trim_cache size 0 max 16384 2023-10-08T09:57:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.050+0000 7f76ea83eb80 1 Processor -- start 2023-10-08T09:57:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 -- start start 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0c678c0 con 0x55b1e0e947a0 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0e906e0 con 0x55b1e0e952b0 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.051+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0e8f500 con 0x55b1e0e93e40 2023-10-08T09:57:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.052+0000 7f76d77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.052+0000 7f76d77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58084/0 (socket says 172.21.15.97:58084) 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.052+0000 7f76d77fe700 1 -- 172.21.15.97:0/1174884116 learned_addr learned my addr 172.21.15.97:0/1174884116 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.052+0000 7f76d7fff700 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 -- 172.21.15.97:0/1174884116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 msgr2=0x55b1e0e94220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 -- 172.21.15.97:0/1174884116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 msgr2=0x55b1e0e8e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 -- 172.21.15.97:0/1174884116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e0c381f0 con 0x55b1e0e947a0 2023-10-08T09:57:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.053+0000 7f76d77fe700 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005fd0 tx=0x7f76cc01a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad649aec9a8f87d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.055+0000 7f76beffd700 1 -- 172.21.15.97:0/1174884116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0164b0 con 0x55b1e0e947a0 2023-10-08T09:57:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.055+0000 7f76beffd700 1 -- 172.21.15.97:0/1174884116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76cc004830 con 0x55b1e0e947a0 2023-10-08T09:57:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.055+0000 7f76beffd700 1 -- 172.21.15.97:0/1174884116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc005280 con 0x55b1e0e947a0 2023-10-08T09:57:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.056+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1174884116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 msgr2=0x55b1e0ea4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.056+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005fd0 tx=0x7f76cc01a320 comp rx=0 tx=0).stop 2023-10-08T09:57:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.062+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1174884116 shutdown_connections 2023-10-08T09:57:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.062+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0e8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.062+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0e94220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.062+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/1174884116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea4dd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.062+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1174884116 >> 172.21.15.97:0/1174884116 conn(0x55b1e0e08200 msgr2=0x55b1e0ed2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.066+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1174884116 shutdown_connections 2023-10-08T09:57:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.067+0000 7f76ea83eb80 1 -- 172.21.15.97:0/1174884116 wait complete. 2023-10-08T09:57:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.067+0000 7f76ea83eb80 1 -- start start 2023-10-08T09:57:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.067+0000 7f76ea83eb80 1 Processor -- start 2023-10-08T09:57:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0ea2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0ea4650 con 0x55b1e0e947a0 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0ea47c0 con 0x55b1e0e952b0 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e0ea4980 con 0x55b1e0e93e40 2023-10-08T09:57:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76ea83eb80 10 client.0 _ll_register_callbacks cb 0x55b1e0be07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76dcbf2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76dcbf2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38058/0 (socket says 172.21.15.97:38058) 2023-10-08T09:57:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.068+0000 7f76dcbf2700 1 -- 172.21.15.97:0/2214888656 learned_addr learned my addr 172.21.15.97:0/2214888656 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 -- 172.21.15.97:0/2214888656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 msgr2=0x55b1e0ea2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0ea2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 -- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 msgr2=0x55b1e0ea4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.069+0000 7f76d77fe700 1 -- 172.21.15.97:0/2214888656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc01aa80 con 0x55b1e0e947a0 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.070+0000 7f76d77fe700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea31a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55b1e0e59bb0 tx=0x7f76cc016360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacb2c13764b84ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.070+0000 7f76b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.070+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc004320 con 0x55b1e0e947a0 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.070+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f76cc0049a0 con 0x55b1e0e947a0 2023-10-08T09:57:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.070+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d000f050 con 0x55b1e0e947a0 2023-10-08T09:57:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.071+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76cc004b60 con 0x55b1e0e947a0 2023-10-08T09:57:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 10 client.16077 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b1e0c10250 con 0x55b1e0e947a0 2023-10-08T09:57:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d000f050 con 0x55b1e0e947a0 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f76cc033070 con 0x55b1e0e947a0 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 20 client.16077 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0f20140 0x55b1e0f22560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 4 client.16077 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.072+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b1e0ddfad0 con 0x55b1e0f20140 2023-10-08T09:57:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.073+0000 7f76d7fff700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0f20140 0x55b1e0f22560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.078+0000 7f76b77fe700 10 client.16077 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:31.078+0000 7f76d7fff700 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0f20140 0x55b1e0f22560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0eac6234744336d server_cookie=82fe4045f8680da in_seq=0 out_seq=0 2023-10-08T09:57:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:31 smithi186 ceph-mon[104393]: pgmap v371: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 541 MiB/s wr, 5.00k op/s 2023-10-08T09:57:31.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:31 smithi097 ceph-mon[99491]: pgmap v371: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 541 MiB/s wr, 5.00k op/s 2023-10-08T09:57:31.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:31 smithi178 ceph-mon[103156]: pgmap v371: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 541 MiB/s wr, 5.00k op/s 2023-10-08T09:57:33.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:57:33.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.396+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/2132270570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac107b40 msgr2=0x7fe5ac107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:33.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.396+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/2132270570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac107b40 0x7fe5ac107f60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009990 tx=0x7fe5a0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.398+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/2132270570 shutdown_connections 2023-10-08T09:57:33.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.398+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/2132270570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac109260 0x7fe5ac10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.398+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/2132270570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac107b40 0x7fe5ac107f60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.398+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/2132270570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 0x7fe5ac108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.398+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/2132270570 >> 172.21.15.97:0/2132270570 conn(0x7fe5ac06d790 msgr2=0x7fe5ac06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:33.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.399+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/2132270570 shutdown_connections 2023-10-08T09:57:33.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.399+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/2132270570 wait complete. 2023-10-08T09:57:33.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 Processor -- start 2023-10-08T09:57:33.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 -- start start 2023-10-08T09:57:33.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 0x7fe5ac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 0x7fe5ac19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 0x7fe5ac078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac079050 con 0x7fe5ac109260 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0791c0 con 0x7fe5ac107b40 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.400+0000 7fe5b2ec7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac079330 con 0x7fe5ac1088a0 2023-10-08T09:57:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 0x7fe5ac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 0x7fe5ac078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 0x7fe5ac19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38072/0 (socket says 172.21.15.97:38072) 2023-10-08T09:57:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b0c63700 1 -- 172.21.15.97:0/3785154455 learned_addr learned my addr 172.21.15.97:0/3785154455 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5abfff700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 0x7fe5ac19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 -- 172.21.15.97:0/3785154455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 msgr2=0x7fe5ac19d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 0x7fe5ac19d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 -- 172.21.15.97:0/3785154455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 msgr2=0x7fe5ac19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 0x7fe5ac19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.402+0000 7fe5b1464700 1 -- 172.21.15.97:0/3785154455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00095f0 con 0x7fe5ac109260 2023-10-08T09:57:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.404+0000 7fe5b1464700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 0x7fe5ac078a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00f780 tx=0x7fe59c00fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.405+0000 7fe5a9ffb700 1 -- 172.21.15.97:0/3785154455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c014650 con 0x7fe5ac109260 2023-10-08T09:57:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.405+0000 7fe5a9ffb700 1 -- 172.21.15.97:0/3785154455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59c00d6c0 con 0x7fe5ac109260 2023-10-08T09:57:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.405+0000 7fe5a9ffb700 1 -- 172.21.15.97:0/3785154455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00bb90 con 0x7fe5ac109260 2023-10-08T09:57:33.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.406+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac079610 con 0x7fe5ac109260 2023-10-08T09:57:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.406+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac079ad0 con 0x7fe5ac109260 2023-10-08T09:57:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.412+0000 7fe5bf0b0b80 1 Processor -- start 2023-10-08T09:57:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 -- start start 2023-10-08T09:57:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5a9ffb700 1 -- 172.21.15.97:0/3785154455 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe59c00bd30 con 0x7fe5ac109260 2023-10-08T09:57:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adf52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adeb9460 con 0x5586adf519c0 2023-10-08T09:57:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adeaac20 con 0x5586adf51060 2023-10-08T09:57:33.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.414+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adea8540 con 0x5586adf52800 2023-10-08T09:57:33.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.415+0000 7fe5a9ffb700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 0x7fe5940a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.415+0000 7fe5a9ffb700 1 -- 172.21.15.97:0/3785154455 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe59c0d0b20 con 0x7fe5ac109260 2023-10-08T09:57:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.415+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58106/0 (socket says 172.21.15.97:58106) 2023-10-08T09:57:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 -- 172.21.15.97:0/1894337558 learned_addr learned my addr 172.21.15.97:0/1894337558 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 -- 172.21.15.97:0/1894337558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 msgr2=0x5586adf52c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adf52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 -- 172.21.15.97:0/1894337558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 msgr2=0x5586adf51480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 -- 172.21.15.97:0/1894337558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586adccede0 con 0x5586adf519c0 2023-10-08T09:57:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.416+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ca0 tx=0x7fe5a000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc26a673ca5c99c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.417+0000 7fe5b1464700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 0x7fe5940a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.424+0000 7fe592ffd700 1 -- 172.21.15.97:0/1894337558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001d070 con 0x5586adf519c0 2023-10-08T09:57:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.424+0000 7fe592ffd700 1 -- 172.21.15.97:0/1894337558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5a001e040 con 0x5586adf519c0 2023-10-08T09:57:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.424+0000 7fe592ffd700 1 -- 172.21.15.97:0/1894337558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0004bf0 con 0x5586adf519c0 2023-10-08T09:57:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.425+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1894337558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 msgr2=0x5586adf52270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.425+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ca0 tx=0x7fe5a000be00 comp rx=0 tx=0).stop 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.428+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1894337558 shutdown_connections 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.428+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adf52c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.428+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.428+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1894337558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf52270 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.428+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1894337558 >> 172.21.15.97:0/1894337558 conn(0x5586adec5260 msgr2=0x5586adf4be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:33.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.429+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1894337558 shutdown_connections 2023-10-08T09:57:33.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.430+0000 7fe5b1464700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 0x7fe5940a3af0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00d400 tx=0x7fe59c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:33.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.430+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1894337558 wait complete. 2023-10-08T09:57:33.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.430+0000 7fe5bf0b0b80 1 -- start start 2023-10-08T09:57:33.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.430+0000 7fe5bf0b0b80 1 Processor -- start 2023-10-08T09:57:33.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf52800 0x5586adf6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adea3e30 con 0x5586adf519c0 2023-10-08T09:57:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586ade40710 con 0x5586adf52800 2023-10-08T09:57:33.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586ade403b0 con 0x5586adf51060 2023-10-08T09:57:33.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5bf0b0b80 10 client.0 _ll_register_callbacks cb 0x5586adc9d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:33.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58114/0 (socket says 172.21.15.97:58114) 2023-10-08T09:57:33.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.431+0000 7fe5b0c63700 1 -- 172.21.15.97:0/698169518 learned_addr learned my addr 172.21.15.97:0/698169518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:33.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.432+0000 7fe5b0c63700 1 -- 172.21.15.97:0/698169518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 msgr2=0x5586adf636d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:57:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.432+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf636d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.432+0000 7fe5b0c63700 1 -- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf52800 msgr2=0x5586adf6ac00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:57:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.432+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf52800 0x5586adf6ac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:33.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.432+0000 7fe5b0c63700 1 -- 172.21.15.97:0/698169518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac110380 con 0x5586adf519c0 2023-10-08T09:57:33.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ce0 tx=0x7fe5a0005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59952cfbbb255755 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:33.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe58bfff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:57:33.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001d070 con 0x5586adf519c0 2023-10-08T09:57:33.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe5a0004e40 con 0x5586adf519c0 2023-10-08T09:57:33.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586ade6beb0 con 0x5586adf519c0 2023-10-08T09:57:33.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5a000f460 con 0x5586adf519c0 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe5bf0b0b80 10 client.16089 fetch_fsmap learned FSMap version 25 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.433+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5586adcb7c60 con 0x5586adf519c0 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586ade6beb0 con 0x5586adf519c0 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe5a003c820 con 0x5586adf519c0 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T09:57:33.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:33.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 20 client.16089 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:33.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adf5d0c0 0x5586adf5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:33.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 4 client.16089 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:57:33.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.434+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586ade9ce10 con 0x5586adf5d0c0 2023-10-08T09:57:33.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.437+0000 7fe5abfff700 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adf5d0c0 0x5586adf5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:33.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.455+0000 7fe5abfff700 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adf5d0c0 0x5586adf5f560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98089a25596026d5 server_cookie=bea16ea3364a767e in_seq=0 out_seq=0 2023-10-08T09:57:33.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:33.456+0000 7fe58bfff700 10 client.16089 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:33.464 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:33 smithi097 ceph-mon[99491]: pgmap v372: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 500 MiB/s wr, 8.10k op/s 2023-10-08T09:57:33.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:33 smithi186 ceph-mon[104393]: pgmap v372: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 500 MiB/s wr, 8.10k op/s 2023-10-08T09:57:33.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:33 smithi178 ceph-mon[103156]: pgmap v372: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 500 MiB/s wr, 8.10k op/s 2023-10-08T09:57:35.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:35 smithi186 ceph-mon[104393]: pgmap v373: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 303 MiB/s wr, 10.44k op/s 2023-10-08T09:57:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:35 smithi178 ceph-mon[103156]: pgmap v373: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 303 MiB/s wr, 10.44k op/s 2023-10-08T09:57:35.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:35 smithi097 ceph-mon[99491]: pgmap v373: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 303 MiB/s wr, 10.44k op/s 2023-10-08T09:57:37.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:37 smithi097 ceph-mon[99491]: pgmap v374: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 213 MiB/s wr, 12.90k op/s 2023-10-08T09:57:37.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:37 smithi178 ceph-mon[103156]: pgmap v374: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 213 MiB/s wr, 12.90k op/s 2023-10-08T09:57:37.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:37 smithi186 ceph-mon[104393]: pgmap v374: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 213 MiB/s wr, 12.90k op/s 2023-10-08T09:57:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:39 smithi178 ceph-mon[103156]: pgmap v375: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 MiB/s wr, 14.36k op/s 2023-10-08T09:57:39.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:39 smithi097 ceph-mon[99491]: pgmap v375: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 MiB/s wr, 14.36k op/s 2023-10-08T09:57:39.779 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:39 smithi186 ceph-mon[104393]: pgmap v375: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 MiB/s wr, 14.36k op/s 2023-10-08T09:57:40.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:40 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:40.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:40 smithi178 ceph-mon[103156]: pgmap v376: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 83 MiB/s wr, 13.20k op/s 2023-10-08T09:57:40.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:40 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:40.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:40 smithi097 ceph-mon[99491]: pgmap v376: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 83 MiB/s wr, 13.20k op/s 2023-10-08T09:57:40.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:40 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:57:40.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:40 smithi186 ceph-mon[104393]: pgmap v376: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 83 MiB/s wr, 13.20k op/s 2023-10-08T09:57:43.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:43 smithi097 ceph-mon[99491]: pgmap v377: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 92 MiB/s wr, 14.60k op/s 2023-10-08T09:57:43.460 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:43 smithi178 ceph-mon[103156]: pgmap v377: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 92 MiB/s wr, 14.60k op/s 2023-10-08T09:57:43.534 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:43 smithi186 ceph-mon[104393]: pgmap v377: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 92 MiB/s wr, 14.60k op/s 2023-10-08T09:57:45.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:45 smithi097 ceph-mon[99491]: pgmap v378: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:45.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:45 smithi178 ceph-mon[103156]: pgmap v378: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:45.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:45 smithi186 ceph-mon[104393]: pgmap v378: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:47.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:47 smithi186 ceph-mon[104393]: pgmap v379: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 MiB/s wr, 13.36k op/s 2023-10-08T09:57:47.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:47 smithi097 ceph-mon[99491]: pgmap v379: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 MiB/s wr, 13.36k op/s 2023-10-08T09:57:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:47 smithi178 ceph-mon[103156]: pgmap v379: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 85 MiB/s wr, 13.36k op/s 2023-10-08T09:57:49.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:49 smithi186 ceph-mon[104393]: pgmap v380: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.85k op/s 2023-10-08T09:57:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:49 smithi178 ceph-mon[103156]: pgmap v380: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.85k op/s 2023-10-08T09:57:49.708 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:49 smithi097 ceph-mon[99491]: pgmap v380: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.85k op/s 2023-10-08T09:57:51.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:51 smithi097 ceph-mon[99491]: pgmap v381: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 MiB/s wr, 12.23k op/s 2023-10-08T09:57:51.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:51 smithi178 ceph-mon[103156]: pgmap v381: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 MiB/s wr, 12.23k op/s 2023-10-08T09:57:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:51 smithi186 ceph-mon[104393]: pgmap v381: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 MiB/s wr, 12.23k op/s 2023-10-08T09:57:53.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:53 smithi178 ceph-mon[103156]: pgmap v382: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:53.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:53 smithi097 ceph-mon[99491]: pgmap v382: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:53.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:53 smithi186 ceph-mon[104393]: pgmap v382: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 86 MiB/s wr, 13.57k op/s 2023-10-08T09:57:55.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:55 smithi097 ceph-mon[99491]: pgmap v383: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:55 smithi178 ceph-mon[103156]: pgmap v383: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:55.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:55 smithi186 ceph-mon[104393]: pgmap v383: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:56.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:56 smithi097 ceph-mon[99491]: pgmap v384: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:56.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:56 smithi178 ceph-mon[103156]: pgmap v384: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:56.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:56 smithi186 ceph-mon[104393]: pgmap v384: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 84 MiB/s wr, 13.40k op/s 2023-10-08T09:57:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.603+0000 7f76b77fe700 1 -- 172.21.15.97:0/2214888656 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b1e0ddfad0 con 0x55b1e0f20140 2023-10-08T09:57:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.603+0000 7f76b77fe700 10 client.16077 handle_command_reply: tid=0 2023-10-08T09:57:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.603+0000 7f76ea83eb80 1 client.16077 shutdown 2023-10-08T09:57:58.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.604+0000 7fe58bfff700 1 -- 172.21.15.97:0/698169518 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5586ade9ce10 con 0x5586adf5d0c0 2023-10-08T09:57:58.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.604+0000 7fe58bfff700 10 client.16089 handle_command_reply: tid=0 2023-10-08T09:57:58.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.604+0000 7fe5bf0b0b80 1 client.16089 shutdown 2023-10-08T09:57:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.607+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 msgr2=0x5586adf63c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.607+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ce0 tx=0x7fe5a0005460 comp rx=0 tx=0).stop 2023-10-08T09:57:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 shutdown_connections 2023-10-08T09:57:58.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf52800 0x5586adf6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adf5d0c0 0x5586adf5f560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/698169518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf63c10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 >> 172.21.15.97:0/698169518 conn(0x5586adec5260 msgr2=0x5586adf4b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.608+0000 7fe58bfff700 0 client.16089 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.605+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 msgr2=0x55b1e0ea31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.605+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea31a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55b1e0e59bb0 tx=0x7f76cc016360 comp rx=0 tx=0).stop 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 shutdown_connections 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1e0e952b0 0x55b1e0ea4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55b1e0f20140 0x55b1e0f22560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b1e0e93e40 0x55b1e0ea2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 --2- 172.21.15.97:0/2214888656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b1e0e947a0 0x55b1e0ea31a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 >> 172.21.15.97:0/2214888656 conn(0x55b1e0e08200 msgr2=0x55b1e0e982c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.608+0000 7f76b77fe700 0 client.16077 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:58.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.609+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 shutdown_connections 2023-10-08T09:57:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.609+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 shutdown_connections 2023-10-08T09:57:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.609+0000 7f76ea83eb80 1 -- 172.21.15.97:0/2214888656 wait complete. 2023-10-08T09:57:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.611+0000 7f76ea83eb80 20 client.16077 trim_cache size 0 max 16384 2023-10-08T09:57:58.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.611+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/698169518 wait complete. 2023-10-08T09:57:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:57:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:57:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:57:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:57:58.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.614+0000 7fe5bf0b0b80 20 client.16089 trim_cache size 0 max 16384 2023-10-08T09:57:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 msgr2=0x7f76c00a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 0x7f76c00a3b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76d0020000 tx=0x7f76d001d000 comp rx=0 tx=0).stop 2023-10-08T09:57:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 msgr2=0x7f76d819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c010 tx=0x7f76c800bbb0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 shutdown_connections 2023-10-08T09:57:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76c00a1650 0x7f76c00a3b10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d810cb30 0x7f76d8078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76d81077f0 0x7f76d819cd60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 --2- 172.21.15.97:0/106105855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d810a160 0x7f76d819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.617+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 >> 172.21.15.97:0/106105855 conn(0x7f76d806d770 msgr2=0x7f76d8108030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.632+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 shutdown_connections 2023-10-08T09:57:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:57:58.639+0000 7f76de655700 1 -- 172.21.15.97:0/106105855 wait complete. 2023-10-08T09:57:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 Processor -- start 2023-10-08T09:57:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 -- start start 2023-10-08T09:57:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf70930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf52800 0x5586adf639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586add24920 con 0x5586adf52800 2023-10-08T09:57:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adf64870 con 0x5586adf519c0 2023-10-08T09:57:58.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.646+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adf52040 con 0x5586adf51060 2023-10-08T09:57:58.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.653+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:58.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.653+0000 7fe5b0c63700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38092/0 (socket says 172.21.15.97:38092) 2023-10-08T09:57:58.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.653+0000 7fe5b0c63700 1 -- 172.21.15.97:0/2183274770 learned_addr learned my addr 172.21.15.97:0/2183274770 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 -- 172.21.15.97:0/2183274770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 msgr2=0x5586adf70930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf70930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 -- 172.21.15.97:0/2183274770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf52800 msgr2=0x5586adf639c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf52800 0x5586adf639c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 -- 172.21.15.97:0/2183274770 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586adcf5250 con 0x5586adf519c0 2023-10-08T09:57:58.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.654+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000bf60 tx=0x7fe5a0049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6ce05ea24bf6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:58.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.655+0000 7fe5937fe700 1 -- 172.21.15.97:0/2183274770 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001d040 con 0x5586adf519c0 2023-10-08T09:57:58.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.658+0000 7fe5937fe700 1 -- 172.21.15.97:0/2183274770 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5a003cd50 con 0x5586adf519c0 2023-10-08T09:57:58.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.658+0000 7fe5937fe700 1 -- 172.21.15.97:0/2183274770 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00477a0 con 0x5586adf519c0 2023-10-08T09:57:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.662+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/2183274770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 msgr2=0x5586adf63480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.662+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000bf60 tx=0x7fe5a0049000 comp rx=0 tx=0).stop 2023-10-08T09:57:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.663+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/2183274770 shutdown_connections 2023-10-08T09:57:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.663+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf52800 0x5586adf639c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.663+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf51060 0x5586adf70930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.663+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/2183274770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf519c0 0x5586adf63480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.663+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/2183274770 >> 172.21.15.97:0/2183274770 conn(0x5586adec5260 msgr2=0x5586adf55800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.675+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/2183274770 shutdown_connections 2023-10-08T09:57:58.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.690+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/2183274770 wait complete. 2023-10-08T09:57:58.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.691+0000 7fe5bf0b0b80 1 -- start start 2023-10-08T09:57:58.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.691+0000 7fe5bf0b0b80 1 Processor -- start 2023-10-08T09:57:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adfb9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adf70700 con 0x5586adf519c0 2023-10-08T09:57:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adfba030 con 0x5586adf51060 2023-10-08T09:57:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586adfba1c0 con 0x5586adf52800 2023-10-08T09:57:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.694+0000 7fe5bf0b0b80 10 client.0 _ll_register_callbacks cb 0x7fe5a0005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:57:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38096/0 (socket says 172.21.15.97:38096) 2023-10-08T09:57:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 -- 172.21.15.97:0/1105796075 learned_addr learned my addr 172.21.15.97:0/1105796075 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 -- 172.21.15.97:0/1105796075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 msgr2=0x5586adfb9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adfb9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 -- 172.21.15.97:0/1105796075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 msgr2=0x5586adf70020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:57:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.698+0000 7fe5abfff700 1 -- 172.21.15.97:0/1105796075 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586adcb7c60 con 0x5586adf51060 2023-10-08T09:57:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.699+0000 7fe5abfff700 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009bd0 tx=0x7fe5a4016b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543f6001c8742398 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.702+0000 7fe58bfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-08T09:57:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.702+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4011040 con 0x5586adf51060 2023-10-08T09:57:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.702+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe5a4006e40 con 0x5586adf51060 2023-10-08T09:57:58.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.702+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586ade9cb30 con 0x5586adf51060 2023-10-08T09:57:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.711+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5a400b750 con 0x5586adf51060 2023-10-08T09:57:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.711+0000 7fe5bf0b0b80 10 client.25502 fetch_fsmap learned FSMap version 26 2023-10-08T09:57:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.711+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5586adccd2b0 con 0x5586adf51060 2023-10-08T09:57:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586ade9cb30 con 0x5586adf51060 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe5a4029b10 con 0x5586adf51060 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 20 client.25502 populate_metadata read hostname 'smithi097' 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adfbbcc0 0x5586adfbe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 4 client.25502 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:57:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.713+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586adceff70 con 0x5586adfbbcc0 2023-10-08T09:57:58.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.725+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adfbbcc0 0x5586adfbe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.726+0000 7fe5b0c63700 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adfbbcc0 0x5586adfbe0e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db41d4f59104a0db server_cookie=4b5ec3e7c7a56b53 in_seq=0 out_seq=0 2023-10-08T09:57:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.728+0000 7fe58bfff700 10 client.25502 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.728+0000 7fe58bfff700 1 -- 172.21.15.97:0/1105796075 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5586adceff70 con 0x5586adfbbcc0 2023-10-08T09:57:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.728+0000 7fe58bfff700 10 client.25502 handle_command_reply: tid=0 2023-10-08T09:57:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.728+0000 7fe5bf0b0b80 1 client.25502 shutdown 2023-10-08T09:57:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.734+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 msgr2=0x5586adf6fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.734+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009bd0 tx=0x7fe5a4016b80 comp rx=0 tx=0).stop 2023-10-08T09:57:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 shutdown_connections 2023-10-08T09:57:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5586adfbbcc0 0x5586adfbe0e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5586adf52800 0x5586adfb9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586adf51060 0x5586adf6fae0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 --2- 172.21.15.97:0/1105796075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5586adf519c0 0x5586adf70020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 >> 172.21.15.97:0/1105796075 conn(0x5586adec5260 msgr2=0x5586adf55800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe58bfff700 0 client.25502 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:57:58.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.740+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 shutdown_connections 2023-10-08T09:57:58.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.747+0000 7fe5bf0b0b80 1 -- 172.21.15.97:0/1105796075 wait complete. 2023-10-08T09:57:58.750 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:57:58.750 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2023-10-08T09:57:58.750 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {} 2023-10-08T09:57:58.750 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:57:58.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.749+0000 7fe5bf0b0b80 20 client.25502 trim_cache size 0 max 16384 2023-10-08T09:57:58.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.759+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 msgr2=0x7fe5940a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.759+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 0x7fe5940a3af0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00d400 tx=0x7fe59c00d040 comp rx=0 tx=0).stop 2023-10-08T09:57:58.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.759+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 msgr2=0x7fe5ac078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:58.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.759+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 0x7fe5ac078a50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00f780 tx=0x7fe59c00fa90 comp rx=0 tx=0).stop 2023-10-08T09:57:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 shutdown_connections 2023-10-08T09:57:58.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe5940a1630 0x7fe5940a3af0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5ac109260 0x7fe5ac078a50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5ac107b40 0x7fe5ac19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 --2- 172.21.15.97:0/3785154455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5ac1088a0 0x7fe5ac19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.762+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 >> 172.21.15.97:0/3785154455 conn(0x7fe5ac06d790 msgr2=0x7fe5ac10bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.768+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 shutdown_connections 2023-10-08T09:57:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:58.770+0000 7fe5b2ec7700 1 -- 172.21.15.97:0/3785154455 wait complete. 2023-10-08T09:57:58.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T09:57:58.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 -- 172.21.15.97:0/601863728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200075420 msgr2=0x7ff2000778b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 --2- 172.21.15.97:0/601863728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200075420 0x7ff2000778b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b970 tx=0x7ff1f000bc80 comp rx=0 tx=0).stop 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 -- 172.21.15.97:0/601863728 shutdown_connections 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 --2- 172.21.15.97:0/601863728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200077df0 0x7ff20007a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 --2- 172.21.15.97:0/601863728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff200072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 --2- 172.21.15.97:0/601863728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200075420 0x7ff2000778b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.080+0000 7ff207715700 1 -- 172.21.15.97:0/601863728 >> 172.21.15.97:0/601863728 conn(0x7ff20006d6c0 msgr2=0x7ff20006fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:59.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.081+0000 7ff207715700 1 -- 172.21.15.97:0/601863728 shutdown_connections 2023-10-08T09:57:59.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.084+0000 7ff207715700 1 -- 172.21.15.97:0/601863728 wait complete. 2023-10-08T09:57:59.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.085+0000 7ff207715700 1 Processor -- start 2023-10-08T09:57:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 -- start start 2023-10-08T09:57:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200075420 0x7ff2001c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200077df0 0x7ff2001c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2001c67d0 con 0x7ff200072540 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2001c6940 con 0x7ff200077df0 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.090+0000 7ff207715700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2001c6b00 con 0x7ff200075420 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58140/0 (socket says 172.21.15.97:58140) 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 -- 172.21.15.97:0/3964529436 learned_addr learned my addr 172.21.15.97:0/3964529436 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff204cb0700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200075420 0x7ff2001c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 -- 172.21.15.97:0/3964529436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200075420 msgr2=0x7ff2001c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200075420 0x7ff2001c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 -- 172.21.15.97:0/3964529436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200077df0 msgr2=0x7ff2001c6030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200077df0 0x7ff2001c6030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.093+0000 7ff2054b1700 1 -- 172.21.15.97:0/3964529436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f000b5a0 con 0x7ff200072540 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.094+0000 7ff2054b1700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc007850 tx=0x7ff1fc007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.094+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00f040 con 0x7ff200072540 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.094+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1fc004cf0 con 0x7ff200072540 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.094+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00d810 con 0x7ff200072540 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.096+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2001c6de0 con 0x7ff200072540 2023-10-08T09:57:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.096+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2001c7290 con 0x7ff200072540 2023-10-08T09:57:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.100+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e4005320 con 0x7ff200072540 2023-10-08T09:57:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.103+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff1fc004750 con 0x7ff200072540 2023-10-08T09:57:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.104+0000 7ff1f67fc700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 0x7ff1ec0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:57:59.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.104+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff1fc0cc1b0 con 0x7ff200072540 2023-10-08T09:57:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.114+0000 7ff204cb0700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 0x7ff1ec0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:57:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.114+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff1fc095480 con 0x7ff200072540 2023-10-08T09:57:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.135+0000 7ff204cb0700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 0x7ff1ec0a3b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009230 tx=0x7ff1f000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:57:59.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:57:59 smithi097 ceph-mon[99491]: pgmap v385: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 12.81k op/s 2023-10-08T09:57:59.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:57:59 smithi178 ceph-mon[103156]: pgmap v385: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 12.81k op/s 2023-10-08T09:57:59.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:57:59 smithi186 ceph-mon[104393]: pgmap v385: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 78 MiB/s wr, 12.81k op/s 2023-10-08T09:57:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T09:57:59.604 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T09:57:39.190072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:57:59.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.602+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1e4005b50 con 0x7ff200072540 2023-10-08T09:57:59.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.603+0000 7ff1f67fc700 1 -- 172.21.15.97:0/3964529436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11340 (secure 0 0 0) 0x7ff1fc017090 con 0x7ff200072540 2023-10-08T09:57:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.615+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 msgr2=0x7ff1ec0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.615+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 0x7ff1ec0a3b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009230 tx=0x7ff1f000d040 comp rx=0 tx=0).stop 2023-10-08T09:57:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.615+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 msgr2=0x7ff2001c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.615+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc007850 tx=0x7ff1fc007c10 comp rx=0 tx=0).stop 2023-10-08T09:57:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 shutdown_connections 2023-10-08T09:57:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff1ec0a1680 0x7ff1ec0a3b40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff200077df0 0x7ff2001c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200072540 0x7ff2001c1530 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 --2- 172.21.15.97:0/3964529436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff200075420 0x7ff2001c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:57:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.618+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 >> 172.21.15.97:0/3964529436 conn(0x7ff20006d6c0 msgr2=0x7ff200076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:57:59.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.631+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 shutdown_connections 2023-10-08T09:57:59.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.632+0000 7ff207715700 1 -- 172.21.15.97:0/3964529436 wait complete. 2023-10-08T09:57:59.657 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2023-10-08T09:57:59.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T09:57:59.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.987+0000 7f16ec99e700 1 -- 172.21.15.97:0/661964184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e810cb30 msgr2=0x7f16e810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:57:59.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.987+0000 7f16ec99e700 1 --2- 172.21.15.97:0/661964184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e810cb30 0x7f16e810cfb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.993+0000 7f16ec99e700 1 -- 172.21.15.97:0/661964184 shutdown_connections 2023-10-08T09:58:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.993+0000 7f16ec99e700 1 --2- 172.21.15.97:0/661964184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e810cb30 0x7f16e810cfb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.993+0000 7f16ec99e700 1 --2- 172.21.15.97:0/661964184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e81077f0 0x7f16e8109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.993+0000 7f16ec99e700 1 --2- 172.21.15.97:0/661964184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:57:59.993+0000 7f16ec99e700 1 -- 172.21.15.97:0/661964184 >> 172.21.15.97:0/661964184 conn(0x7f16e806d790 msgr2=0x7f16e806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.003+0000 7f16ec99e700 1 -- 172.21.15.97:0/661964184 shutdown_connections 2023-10-08T09:58:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.009+0000 7f16ec99e700 1 -- 172.21.15.97:0/661964184 wait complete. 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 Processor -- start 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 -- start start 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 0x7f16e8113d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e81142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e81172f0 con 0x7f16e810cb30 2023-10-08T09:58:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8117460 con 0x7f16e81077f0 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.010+0000 7f16ec99e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e81b5c20 con 0x7f16e810a160 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.011+0000 7f16e6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.011+0000 7f16e6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58152/0 (socket says 172.21.15.97:58152) 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.011+0000 7f16e6d9d700 1 -- 172.21.15.97:0/1506991888 learned_addr learned my addr 172.21.15.97:0/1506991888 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.011+0000 7f16e659c700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 0x7f16e8113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.013+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e81142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 -- 172.21.15.97:0/1506991888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 msgr2=0x7f16e81142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e81142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 -- 172.21.15.97:0/1506991888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 msgr2=0x7f16e8113d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 0x7f16e8113d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 -- 172.21.15.97:0/1506991888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0095f0 con 0x7f16e810cb30 2023-10-08T09:58:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005910 tx=0x7f16dc00f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16d77fe700 1 -- 172.21.15.97:0/1506991888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01d070 con 0x7f16e810cb30 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e81b5d60 con 0x7f16e810cb30 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16d77fe700 1 -- 172.21.15.97:0/1506991888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc004cf0 con 0x7f16e810cb30 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16d77fe700 1 -- 172.21.15.97:0/1506991888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005600 con 0x7f16e810cb30 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e81b60d0 con 0x7f16e810cb30 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.017+0000 7f16e659c700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 0x7f16e8113d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.021+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.022+0000 7f16f8b87b80 1 Processor -- start 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.022+0000 7f16f8b87b80 1 -- start start 2023-10-08T09:58:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.022+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fb3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.022+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fb4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.022+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fb4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.023+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446f1b460 con 0x556446fb39c0 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.023+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446f0cc20 con 0x556446fb4800 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.023+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446f0a540 con 0x556446fb3060 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.029+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fb3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.029+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fb4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.029+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fb3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59858/0 (socket says 172.21.15.97:59858) 2023-10-08T09:58:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.029+0000 7f16e5d9b700 1 -- 172.21.15.97:0/57544541 learned_addr learned my addr 172.21.15.97:0/57544541 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.033+0000 7f16e5d9b700 1 -- 172.21.15.97:0/57544541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 msgr2=0x556446fb3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.033+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fb3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.033+0000 7f16e5d9b700 1 -- 172.21.15.97:0/57544541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 msgr2=0x556446fb4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.033+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fb4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.033+0000 7f16e5d9b700 1 -- 172.21.15.97:0/57544541 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556446d30de0 con 0x556446fb4800 2023-10-08T09:58:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.035+0000 7f16d77fe700 1 -- 172.21.15.97:0/1506991888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f16dc00fbc0 con 0x7f16e810cb30 2023-10-08T09:58:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.035+0000 7f16d77fe700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 0x7f16d00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.036+0000 7f16d77fe700 1 -- 172.21.15.97:0/1506991888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f16dc0ce2f0 con 0x7f16e810cb30 2023-10-08T09:58:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.039+0000 7f16e659c700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 0x7f16d00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.040+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fb4c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16d80176a0 tx=0x7f16d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614319da73a1b306 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.040+0000 7f16d4ff9700 1 -- 172.21.15.97:0/57544541 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800c070 con 0x556446fb4800 2023-10-08T09:58:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.040+0000 7f16d4ff9700 1 -- 172.21.15.97:0/57544541 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d8017ed0 con 0x556446fb4800 2023-10-08T09:58:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.040+0000 7f16d4ff9700 1 -- 172.21.15.97:0/57544541 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d80136c0 con 0x556446fb4800 2023-10-08T09:58:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.043+0000 7f16e659c700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 0x7f16d00a3a90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x556446fb3e00 tx=0x7f16e0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.047+0000 7f16f8b87b80 1 -- 172.21.15.97:0/57544541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 msgr2=0x556446fb4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.047+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fb4c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16d80176a0 tx=0x7f16d8004750 comp rx=0 tx=0).stop 2023-10-08T09:58:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.049+0000 7f16f8b87b80 1 -- 172.21.15.97:0/57544541 shutdown_connections 2023-10-08T09:58:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.049+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fb4c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.049+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fb3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.049+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/57544541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fb4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.049+0000 7f16f8b87b80 1 -- 172.21.15.97:0/57544541 >> 172.21.15.97:0/57544541 conn(0x556446f27260 msgr2=0x556446fade00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.056+0000 7f16f8b87b80 1 -- 172.21.15.97:0/57544541 shutdown_connections 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.059+0000 7f16f8b87b80 1 -- 172.21.15.97:0/57544541 wait complete. 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.060+0000 7f16f8b87b80 1 -- start start 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.060+0000 7f16f8b87b80 1 Processor -- start 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb3060 0x556446fbcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446f05e30 con 0x556446fb3060 2023-10-08T09:58:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446ea2710 con 0x556446fb4800 2023-10-08T09:58:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446ea23b0 con 0x556446fb39c0 2023-10-08T09:58:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.061+0000 7f16f8b87b80 10 client.0 _ll_register_callbacks cb 0x556446cff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:00.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59866/0 (socket says 172.21.15.97:59866) 2023-10-08T09:58:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 -- 172.21.15.97:0/4148207529 learned_addr learned my addr 172.21.15.97:0/4148207529 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e659c700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb3060 0x556446fbcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 -- 172.21.15.97:0/4148207529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 msgr2=0x556446fc4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fc4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 -- 172.21.15.97:0/4148207529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb3060 msgr2=0x556446fbcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb3060 0x556446fbcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.064+0000 7f16e5d9b700 1 -- 172.21.15.97:0/4148207529 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8017380 con 0x556446fb39c0 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009230 tx=0x7f16d80189f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9413c946edf523c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:58:00.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800c040 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f16d8011040 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556446ecdeb0 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.065+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d8009b60 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.066+0000 7f16f8b87b80 10 client.25480 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.066+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556446d19c60 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.066+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556446ecdeb0 con 0x556446fb39c0 2023-10-08T09:58:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.066+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f16d802a070 con 0x556446fb39c0 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 10 client.25480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 20 client.25480 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x556446feab00 0x556446fecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 4 client.25480 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.067+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556446efee10 con 0x556446feab00 2023-10-08T09:58:00.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.070+0000 7f16e659c700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x556446feab00 0x556446fecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.084+0000 7f16cdffb700 10 client.25480 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:58:00.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.084+0000 7f16e659c700 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x556446feab00 0x556446fecf20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b4801d83bc4bce8 server_cookie=8f0d3e25a0a3859e in_seq=0 out_seq=0 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.088+0000 7f16cdffb700 1 -- 172.21.15.97:0/4148207529 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556446efee10 con 0x556446feab00 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.088+0000 7f16cdffb700 10 client.25480 handle_command_reply: tid=0 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.088+0000 7f16f8b87b80 1 client.25480 shutdown 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.088+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 msgr2=0x556446fbd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.088+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009230 tx=0x7f16d80189f0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 shutdown_connections 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x556446feab00 0x556446fecf20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb4800 0x556446fc4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb3060 0x556446fbcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/4148207529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb39c0 0x556446fbd440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 >> 172.21.15.97:0/4148207529 conn(0x556446f27260 msgr2=0x556446fad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.089+0000 7f16cdffb700 0 client.25480 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:58:00.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.092+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 shutdown_connections 2023-10-08T09:58:00.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.092+0000 7f16f8b87b80 1 -- 172.21.15.97:0/4148207529 wait complete. 2023-10-08T09:58:00.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.100+0000 7f16f8b87b80 20 client.25480 trim_cache size 0 max 16384 2023-10-08T09:58:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 Processor -- start 2023-10-08T09:58:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 -- start start 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb3060 0x556446fb4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb4800 0x556446fc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446d86920 con 0x556446fb39c0 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446fc5600 con 0x556446fb3060 2023-10-08T09:58:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446fbca70 con 0x556446fb4800 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.124+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58176/0 (socket says 172.21.15.97:58176) 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 -- 172.21.15.97:0/938590087 learned_addr learned my addr 172.21.15.97:0/938590087 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb4800 0x556446fc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e659c700 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb3060 0x556446fb4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 -- 172.21.15.97:0/938590087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb4800 msgr2=0x556446fc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb4800 0x556446fc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 -- 172.21.15.97:0/938590087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb3060 msgr2=0x556446fb4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb3060 0x556446fb4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.125+0000 7f16e5d9b700 1 -- 172.21.15.97:0/938590087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556446d57250 con 0x556446fb39c0 2023-10-08T09:58:00.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.126+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f16d80198e0 tx=0x7f16d80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d361acf0178316 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.130+0000 7f16d57fa700 1 -- 172.21.15.97:0/938590087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800c040 con 0x556446fb39c0 2023-10-08T09:58:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.130+0000 7f16d57fa700 1 -- 172.21.15.97:0/938590087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d8009e90 con 0x556446fb39c0 2023-10-08T09:58:00.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.130+0000 7f16d57fa700 1 -- 172.21.15.97:0/938590087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d802ab80 con 0x556446fb39c0 2023-10-08T09:58:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.131+0000 7f16f8b87b80 1 -- 172.21.15.97:0/938590087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 msgr2=0x556446fbc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.131+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f16d80198e0 tx=0x7f16d80046d0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.132+0000 7f16f8b87b80 1 -- 172.21.15.97:0/938590087 shutdown_connections 2023-10-08T09:58:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.132+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb4800 0x556446fc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.132+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb3060 0x556446fb4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.132+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/938590087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb39c0 0x556446fbc410 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.132+0000 7f16f8b87b80 1 -- 172.21.15.97:0/938590087 >> 172.21.15.97:0/938590087 conn(0x556446f27260 msgr2=0x556446fa9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.137+0000 7f16f8b87b80 1 -- 172.21.15.97:0/938590087 shutdown_connections 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.140+0000 7f16f8b87b80 1 -- 172.21.15.97:0/938590087 wait complete. 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 -- start start 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 Processor -- start 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fc23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 0x556446fc2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb4800 0x556446fc3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446fc3e80 con 0x556446fb4800 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446fc3ff0 con 0x556446fb39c0 2023-10-08T09:58:00.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556446fc41b0 con 0x556446fb3060 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.141+0000 7f16f8b87b80 10 client.0 _ll_register_callbacks cb 0x7f16d8009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fc23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fc23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59880/0 (socket says 172.21.15.97:59880) 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e659c700 1 -- 172.21.15.97:0/1112879192 learned_addr learned my addr 172.21.15.97:0/1112879192 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb4800 0x556446fc3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 0x556446fc2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 -- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 msgr2=0x556446fc23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fc23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 -- 172.21.15.97:0/1112879192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb4800 msgr2=0x556446fc3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb4800 0x556446fc3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.142+0000 7f16e5d9b700 1 -- 172.21.15.97:0/1112879192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556446d19c60 con 0x556446fb39c0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.143+0000 7f16e5d9b700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 0x556446fc2930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16d8017c90 tx=0x7f16d80189f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be76a889468de4ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.143+0000 7f16cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.144+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800c040 con 0x556446fb39c0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.144+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f16d8000e40 con 0x556446fb39c0 2023-10-08T09:58:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.144+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16d8017ed0 con 0x556446fb39c0 2023-10-08T09:58:00.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.144+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d801aef0 con 0x556446fb39c0 2023-10-08T09:58:00.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.147+0000 7f16f8b87b80 10 client.25523 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:00.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.147+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556446d2f2b0 con 0x556446fb39c0 2023-10-08T09:58:00.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d8046080 con 0x556446fb39c0 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f16d80179b0 con 0x556446fb39c0 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 10 client.25523 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 10 client.25523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 20 client.25523 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55644703f5e0 0x556447041a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 4 client.25523 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:58:00.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.148+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 --> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556446efeb30 con 0x55644703f5e0 2023-10-08T09:58:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.153+0000 7f16e659c700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55644703f5e0 0x556447041a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.159+0000 7f16e659c700 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55644703f5e0 0x556447041a00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=defedd0ae0cbfb68 server_cookie=2363717a9dd12482 in_seq=0 out_seq=0 2023-10-08T09:58:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.160+0000 7f16cdffb700 10 client.25523 ms_handle_connect on v2:172.21.15.186:6836/4190406762 2023-10-08T09:58:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.161+0000 7f16cdffb700 1 -- 172.21.15.97:0/1112879192 <== mds.2 v2:172.21.15.186:6836/4190406762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556446efeb30 con 0x55644703f5e0 2023-10-08T09:58:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.161+0000 7f16cdffb700 10 client.25523 handle_command_reply: tid=0 2023-10-08T09:58:00.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.161+0000 7f16f8b87b80 1 client.25523 shutdown 2023-10-08T09:58:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.162+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 msgr2=0x556446fc2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.162+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 0x556446fc2930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16d8017c90 tx=0x7f16d80189f0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 shutdown_connections 2023-10-08T09:58:00.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:6836/4190406762,v1:172.21.15.186:6837/4190406762] conn(0x55644703f5e0 0x556447041a00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556446fb4800 0x556446fc3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556446fb3060 0x556446fc23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 --2- 172.21.15.97:0/1112879192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556446fb39c0 0x556446fc2930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 >> 172.21.15.97:0/1112879192 conn(0x556446f27260 msgr2=0x556446fbcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16cdffb700 0 client.25523 ms_handle_reset on v2:172.21.15.186:6836/4190406762 2023-10-08T09:58:00.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.168+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 shutdown_connections 2023-10-08T09:58:00.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.178+0000 7f16f8b87b80 1 -- 172.21.15.97:0/1112879192 wait complete. 2023-10-08T09:58:00.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.184+0000 7f16f8b87b80 20 client.25523 trim_cache size 0 max 16384 2023-10-08T09:58:00.191 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 msgr2=0x7f16d00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 0x7f16d00a3a90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x556446fb3e00 tx=0x7f16e0009040 comp rx=0 tx=0).stop 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 msgr2=0x7f16e81b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005910 tx=0x7f16dc00f6e0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 shutdown_connections 2023-10-08T09:58:00.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16d00a15d0 0x7f16d00a3a90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e810cb30 0x7f16e81b56e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16e81077f0 0x7f16e8113d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 --2- 172.21.15.97:0/1506991888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16e810a160 0x7f16e81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.194+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 >> 172.21.15.97:0/1506991888 conn(0x7f16e806d790 msgr2=0x7f16e81099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.202+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 shutdown_connections 2023-10-08T09:58:00.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.204+0000 7f16ec99e700 1 -- 172.21.15.97:0/1506991888 wait complete. 2023-10-08T09:58:00.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T09:58:00.352 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:00 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3964529436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:58:00.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:00 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3964529436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:58:00.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:00 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3964529436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:58:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.553+0000 7feabc922700 1 -- 172.21.15.97:0/2200766709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab81080f0 msgr2=0x7feab81085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.553+0000 7feabc922700 1 --2- 172.21.15.97:0/2200766709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab81080f0 0x7feab81085a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feab0009f90 tx=0x7feab000b330 comp rx=0 tx=0).stop 2023-10-08T09:58:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.571+0000 7feabc922700 1 -- 172.21.15.97:0/2200766709 shutdown_connections 2023-10-08T09:58:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.571+0000 7feabc922700 1 --2- 172.21.15.97:0/2200766709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab81080f0 0x7feab81085a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.571+0000 7feabc922700 1 --2- 172.21.15.97:0/2200766709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab80723e0 0x7feab81079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.571+0000 7feabc922700 1 --2- 172.21.15.97:0/2200766709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab810b380 0x7feab810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.571+0000 7feabc922700 1 -- 172.21.15.97:0/2200766709 >> 172.21.15.97:0/2200766709 conn(0x7feab806d790 msgr2=0x7feab806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.573+0000 7feabc922700 1 -- 172.21.15.97:0/2200766709 shutdown_connections 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.573+0000 7feabc922700 1 -- 172.21.15.97:0/2200766709 wait complete. 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 Processor -- start 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 -- start start 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab80723e0 0x7feab819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab810b380 0x7feab81a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81a4a20 con 0x7feab81080f0 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81a4b90 con 0x7feab810b380 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.574+0000 7feabc922700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81a4d00 con 0x7feab80723e0 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.575+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.575+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58196/0 (socket says 172.21.15.97:58196) 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.575+0000 7feab5d9b700 1 -- 172.21.15.97:0/2068524904 learned_addr learned my addr 172.21.15.97:0/2068524904 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 -- 172.21.15.97:0/2068524904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab80723e0 msgr2=0x7feab819ce80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:58:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab80723e0 0x7feab819ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 -- 172.21.15.97:0/2068524904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab810b380 msgr2=0x7feab81a4420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:58:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab810b380 0x7feab81a4420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 -- 172.21.15.97:0/2068524904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0009c40 con 0x7feab81080f0 2023-10-08T09:58:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.576+0000 7feab5d9b700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b500 tx=0x7feaa800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.581+0000 7feaa77fe700 1 -- 172.21.15.97:0/2068524904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8011830 con 0x7feab81080f0 2023-10-08T09:58:00.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.581+0000 7feaa77fe700 1 -- 172.21.15.97:0/2068524904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa8011e50 con 0x7feab81080f0 2023-10-08T09:58:00.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.581+0000 7feaa77fe700 1 -- 172.21.15.97:0/2068524904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800f2e0 con 0x7feab81080f0 2023-10-08T09:58:00.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.582+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab81a4fe0 con 0x7feab81080f0 2023-10-08T09:58:00.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.582+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab81a54a0 con 0x7feab81080f0 2023-10-08T09:58:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.589+0000 7feac8b0bb80 1 Processor -- start 2023-10-08T09:58:00.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.593+0000 7feaa77fe700 1 -- 172.21.15.97:0/2068524904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feaa8011990 con 0x7feab81080f0 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.594+0000 7feaa77fe700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 0x7feaa00a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.594+0000 7feaa77fe700 1 -- 172.21.15.97:0/2068524904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feaa80cc970 con 0x7feab81080f0 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.594+0000 7feab6d9d700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 0x7feaa00a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.594+0000 7feac8b0bb80 1 -- start start 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b24480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a8c460 con 0x55b648b24060 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a7dc20 con 0x55b648b25800 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a7b540 con 0x55b648b249c0 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59902/0 (socket says 172.21.15.97:59902) 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 -- 172.21.15.97:0/1436841362 learned_addr learned my addr 172.21.15.97:0/1436841362 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab659c700 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b24480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 -- 172.21.15.97:0/1436841362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 msgr2=0x55b648b25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 -- 172.21.15.97:0/1436841362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 msgr2=0x55b648b24480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b24480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.595+0000 7feab5d9b700 1 -- 172.21.15.97:0/1436841362 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6488a1de0 con 0x55b648b249c0 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.597+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feaa8099600 tx=0x7feaa8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb440d06745cff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.597+0000 7feaa4ff9700 1 -- 172.21.15.97:0/1436841362 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa809da50 con 0x55b648b249c0 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.597+0000 7feaa4ff9700 1 -- 172.21.15.97:0/1436841362 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa80a1c90 con 0x55b648b249c0 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.597+0000 7feaa4ff9700 1 -- 172.21.15.97:0/1436841362 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8098df0 con 0x55b648b249c0 2023-10-08T09:58:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.598+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1436841362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 msgr2=0x55b648b25270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.598+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feaa8099600 tx=0x7feaa8000bc0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.600+0000 7feab6d9d700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 0x7feaa00a3b10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feab0009f90 tx=0x7feab0008bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1436841362 shutdown_connections 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b24480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1436841362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b25270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1436841362 >> 172.21.15.97:0/1436841362 conn(0x55b648a98260 msgr2=0x55b648b1ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1436841362 shutdown_connections 2023-10-08T09:58:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1436841362 wait complete. 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 -- start start 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.601+0000 7feac8b0bb80 1 Processor -- start 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b24060 0x55b648b2de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a76e30 con 0x55b648b25800 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a13710 con 0x55b648b24060 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a133b0 con 0x55b648b249c0 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feac8b0bb80 10 client.0 _ll_register_callbacks cb 0x55b648870800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feab6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feab6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58212/0 (socket says 172.21.15.97:58212) 2023-10-08T09:58:00.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feab6d9d700 1 -- 172.21.15.97:0/2025250041 learned_addr learned my addr 172.21.15.97:0/2025250041 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.602+0000 7feab5d9b700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 -- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 msgr2=0x55b648b2e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b2e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 -- 172.21.15.97:0/2025250041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b24060 msgr2=0x55b648b2de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b24060 0x55b648b2de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 -- 172.21.15.97:0/2025250041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab81a54a0 con 0x55b648b25800 2023-10-08T09:58:00.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.603+0000 7feab6d9d700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feab0009c10 tx=0x7feab000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705ba6cde0db5d0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.605+0000 7fea9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:00.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.605+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab000e040 con 0x55b648b25800 2023-10-08T09:58:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.605+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feab0007710 con 0x55b648b25800 2023-10-08T09:58:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.605+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b648a3eeb0 con 0x55b648b25800 2023-10-08T09:58:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.606+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab001d440 con 0x55b648b25800 2023-10-08T09:58:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.610+0000 7feac8b0bb80 10 client.16116 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:00.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.610+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b64888ac60 con 0x55b648b25800 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b648a3eeb0 con 0x55b648b25800 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7feab00052f0 con 0x55b648b25800 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 20 client.16116 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:00.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648b5bd00 0x55b648b5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 4 client.16116 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:00.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.625+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b648a6fe10 con 0x55b648b5bd00 2023-10-08T09:58:00.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.635+0000 7feab659c700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648b5bd00 0x55b648b5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.639+0000 7feab659c700 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648b5bd00 0x55b648b5e140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3bfe648604fa6192 server_cookie=3beb3f9333899be6 in_seq=0 out_seq=0 2023-10-08T09:58:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.639+0000 7fea9dffb700 10 client.16116 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:58:00.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.640+0000 7fea9dffb700 1 -- 172.21.15.97:0/2025250041 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b648a6fe10 con 0x55b648b5bd00 2023-10-08T09:58:00.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.640+0000 7fea9dffb700 10 client.16116 handle_command_reply: tid=0 2023-10-08T09:58:00.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.640+0000 7feac8b0bb80 1 client.16116 shutdown 2023-10-08T09:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.655+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 msgr2=0x55b648b35380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.655+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feab0009c10 tx=0x7feab000b330 comp rx=0 tx=0).stop 2023-10-08T09:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 shutdown_connections 2023-10-08T09:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b25800 0x55b648b35380 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648b5bd00 0x55b648b5e140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b24060 0x55b648b2de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/2025250041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b249c0 0x55b648b2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.665+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 >> 172.21.15.97:0/2025250041 conn(0x55b648a98260 msgr2=0x55b648b1e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.671+0000 7fea9dffb700 0 client.16116 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:58:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.679+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 shutdown_connections 2023-10-08T09:58:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.682+0000 7feac8b0bb80 1 -- 172.21.15.97:0/2025250041 wait complete. 2023-10-08T09:58:00.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.684+0000 7feac8b0bb80 20 client.16116 trim_cache size 0 max 16384 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 Processor -- start 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 -- start start 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b25800 0x55b648b28b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6488f7920 con 0x55b648b24060 2023-10-08T09:58:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a6fe10 con 0x55b648b249c0 2023-10-08T09:58:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.710+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a3eeb0 con 0x55b648b25800 2023-10-08T09:58:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.711+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.711+0000 7feab5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38188/0 (socket says 172.21.15.97:38188) 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.711+0000 7feab5d9b700 1 -- 172.21.15.97:0/476048421 learned_addr learned my addr 172.21.15.97:0/476048421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.711+0000 7feab659c700 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.711+0000 7feab6d9d700 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b25800 0x55b648b28b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.712+0000 7feab5d9b700 1 -- 172.21.15.97:0/476048421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b25800 msgr2=0x55b648b28b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.712+0000 7feab5d9b700 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b25800 0x55b648b28b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.712+0000 7feab5d9b700 1 -- 172.21.15.97:0/476048421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 msgr2=0x55b648b28620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.712+0000 7feab5d9b700 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b28620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.712+0000 7feab5d9b700 1 -- 172.21.15.97:0/476048421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6488c8250 con 0x55b648b249c0 2023-10-08T09:58:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.725+0000 7feab5d9b700 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaa800bfd0 tx=0x7feaa809a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebdbbe121fcbeb45 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.727+0000 7feaa57fa700 1 -- 172.21.15.97:0/476048421 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa809da50 con 0x55b648b249c0 2023-10-08T09:58:00.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.727+0000 7feaa57fa700 1 -- 172.21.15.97:0/476048421 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa80b2170 con 0x55b648b249c0 2023-10-08T09:58:00.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.727+0000 7feaa57fa700 1 -- 172.21.15.97:0/476048421 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8098df0 con 0x55b648b249c0 2023-10-08T09:58:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.728+0000 7feac8b0bb80 1 -- 172.21.15.97:0/476048421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 msgr2=0x55b648b35340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.728+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaa800bfd0 tx=0x7feaa809a130 comp rx=0 tx=0).stop 2023-10-08T09:58:00.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.732+0000 7feac8b0bb80 1 -- 172.21.15.97:0/476048421 shutdown_connections 2023-10-08T09:58:00.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.732+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b25800 0x55b648b28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.732+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b24060 0x55b648b28620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.732+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/476048421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b249c0 0x55b648b35340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.732+0000 7feac8b0bb80 1 -- 172.21.15.97:0/476048421 >> 172.21.15.97:0/476048421 conn(0x55b648a98260 msgr2=0x55b648b2bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.733+0000 7feac8b0bb80 1 -- 172.21.15.97:0/476048421 shutdown_connections 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.733+0000 7feac8b0bb80 1 -- 172.21.15.97:0/476048421 wait complete. 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.733+0000 7feac8b0bb80 1 -- start start 2023-10-08T09:58:00.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.733+0000 7feac8b0bb80 1 Processor -- start 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b249c0 0x55b648b31a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b34fb0 con 0x55b648b249c0 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b35120 con 0x55b648b25800 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b352e0 con 0x55b648b24060 2023-10-08T09:58:00.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feac8b0bb80 10 client.0 _ll_register_callbacks cb 0x7feab000bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feab659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feab659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59922/0 (socket says 172.21.15.97:59922) 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feab659c700 1 -- 172.21.15.97:0/1663304476 learned_addr learned my addr 172.21.15.97:0/1663304476 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.734+0000 7feab6d9d700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b249c0 0x55b648b31a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 -- 172.21.15.97:0/1663304476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 msgr2=0x55b648b34a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b34a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 -- 172.21.15.97:0/1663304476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b249c0 msgr2=0x55b648b31a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b249c0 0x55b648b31a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 -- 172.21.15.97:0/1663304476 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6488a1de0 con 0x55b648b24060 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7feab659c700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feab819e380 tx=0x7feaac00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca65bec09210751 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.735+0000 7fea9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.736+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac018c20 con 0x55b648b24060 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.736+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feaac018540 con 0x55b648b24060 2023-10-08T09:58:00.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.736+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa80a1c90 con 0x55b648b24060 2023-10-08T09:58:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaac018d80 con 0x55b648b24060 2023-10-08T09:58:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7feac8b0bb80 10 client.25501 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b6488a02b0 con 0x55b648b24060 2023-10-08T09:58:00.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa80a1c90 con 0x55b648b24060 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7feaac02a520 con 0x55b648b24060 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7feac8b0bb80 10 client.25501 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7feac8b0bb80 10 client.25501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.737+0000 7feac8b0bb80 20 client.25501 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.738+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648bb1070 0x55b648bb3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.738+0000 7feac8b0bb80 4 client.25501 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:58:00.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.738+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 --> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b648a6fb30 con 0x55b648bb1070 2023-10-08T09:58:00.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.742+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648bb1070 0x55b648bb3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:00.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.744+0000 7fea9dffb700 10 client.25501 ms_handle_connect on v2:172.21.15.186:6838/2120449823 2023-10-08T09:58:00.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.744+0000 7feab5d9b700 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648bb1070 0x55b648bb3490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75ffad23f18339b0 server_cookie=e98e95c779e023e0 in_seq=0 out_seq=0 2023-10-08T09:58:00.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.745+0000 7fea9dffb700 1 -- 172.21.15.97:0/1663304476 <== mds.1 v2:172.21.15.186:6838/2120449823 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b648a6fb30 con 0x55b648bb1070 2023-10-08T09:58:00.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.745+0000 7fea9dffb700 10 client.25501 handle_command_reply: tid=0 2023-10-08T09:58:00.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.749+0000 7feac8b0bb80 1 client.25501 shutdown 2023-10-08T09:58:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.751+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 msgr2=0x55b648b31510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.751+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feab819e380 tx=0x7feaac00be60 comp rx=0 tx=0).stop 2023-10-08T09:58:00.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 shutdown_connections 2023-10-08T09:58:00.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:6838/2120449823,v1:172.21.15.186:6839/2120449823] conn(0x55b648bb1070 0x55b648bb3490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b648b25800 0x55b648b34a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b648b24060 0x55b648b31510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 --2- 172.21.15.97:0/1663304476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b648b249c0 0x55b648b31a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 >> 172.21.15.97:0/1663304476 conn(0x55b648a98260 msgr2=0x55b648b28620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.765+0000 7fea9dffb700 0 client.25501 ms_handle_reset on v2:172.21.15.186:6838/2120449823 2023-10-08T09:58:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.773+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 shutdown_connections 2023-10-08T09:58:00.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.773+0000 7feac8b0bb80 1 -- 172.21.15.97:0/1663304476 wait complete. 2023-10-08T09:58:00.784 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:58:00.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.781+0000 7feac8b0bb80 20 client.25501 trim_cache size 0 max 16384 2023-10-08T09:58:00.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.791+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 msgr2=0x7feaa00a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.791+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 0x7feaa00a3b10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feab0009f90 tx=0x7feab0008bf0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.791+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 msgr2=0x7feab819d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:00.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.791+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b500 tx=0x7feaa800b8c0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 shutdown_connections 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7feaa00a1650 0x7feaa00a3b10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feab810b380 0x7feab81a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feab80723e0 0x7feab819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 --2- 172.21.15.97:0/2068524904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab81080f0 0x7feab819d3c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.793+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 >> 172.21.15.97:0/2068524904 conn(0x7feab806d790 msgr2=0x7feab810cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:00.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.799+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 shutdown_connections 2023-10-08T09:58:00.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:00.804+0000 7feabc922700 1 -- 172.21.15.97:0/2068524904 wait complete. 2023-10-08T09:58:00.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T09:58:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.117+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3199444709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc681064d0 msgr2=0x7fdc68108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.117+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3199444709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc681064d0 0x7fdc68108900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000b0a0 tx=0x7fdc6000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.124+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3199444709 shutdown_connections 2023-10-08T09:58:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.124+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3199444709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc681064d0 0x7fdc68108900 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.124+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3199444709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc68101270 0x7fdc68103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.124+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3199444709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68103ba0 0x7fdc68105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.124+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3199444709 >> 172.21.15.97:0/3199444709 conn(0x7fdc680fac40 msgr2=0x7fdc680fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.134+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3199444709 shutdown_connections 2023-10-08T09:58:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.135+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3199444709 wait complete. 2023-10-08T09:58:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.136+0000 7fdc6d9c5700 1 Processor -- start 2023-10-08T09:58:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 -- start start 2023-10-08T09:58:01.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68101270 0x7fdc6806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 0x7fdc6806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 0x7fdc68195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68072670 con 0x7fdc68103ba0 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680727e0 con 0x7fdc68101270 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.137+0000 7fdc6d9c5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68195db0 con 0x7fdc681064d0 2023-10-08T09:58:01.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.139+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 0x7fdc6806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.139+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 0x7fdc6806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58238/0 (socket says 172.21.15.97:58238) 2023-10-08T09:58:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.139+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/3495938138 learned_addr learned my addr 172.21.15.97:0/3495938138 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.140+0000 7fdc677fe700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 0x7fdc68195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.141+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68101270 0x7fdc6806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.142+0000 7fdc677fe700 1 -- 172.21.15.97:0/3495938138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68101270 msgr2=0x7fdc6806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.142+0000 7fdc677fe700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68101270 0x7fdc6806f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.142+0000 7fdc677fe700 1 -- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 msgr2=0x7fdc6806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.142+0000 7fdc677fe700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 0x7fdc6806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.142+0000 7fdc677fe700 1 -- 172.21.15.97:0/3495938138 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009d00 con 0x7fdc681064d0 2023-10-08T09:58:01.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.143+0000 7fdc677fe700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 0x7fdc68195870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009750 tx=0x7fdc600079e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.150+0000 7fdc6c9c3700 1 -- 172.21.15.97:0/3495938138 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000e040 con 0x7fdc681064d0 2023-10-08T09:58:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.150+0000 7fdc6c9c3700 1 -- 172.21.15.97:0/3495938138 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc60005040 con 0x7fdc681064d0 2023-10-08T09:58:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.150+0000 7fdc6c9c3700 1 -- 172.21.15.97:0/3495938138 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60012650 con 0x7fdc681064d0 2023-10-08T09:58:01.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.151+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc680ff060 con 0x7fdc681064d0 2023-10-08T09:58:01.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.151+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc680ff520 con 0x7fdc681064d0 2023-10-08T09:58:01.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.157+0000 7fdc6c9c3700 1 -- 172.21.15.97:0/3495938138 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdc60004a90 con 0x7fdc681064d0 2023-10-08T09:58:01.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.158+0000 7fdc79baeb80 1 Processor -- start 2023-10-08T09:58:01.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.159+0000 7fdc6c9c3700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 0x7fdc540a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.159+0000 7fdc6c9c3700 1 -- 172.21.15.97:0/3495938138 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc60026030 con 0x7fdc681064d0 2023-10-08T09:58:01.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.159+0000 7fdc79baeb80 1 -- start start 2023-10-08T09:58:01.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 0x557ed4e91480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4e92270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 0x557ed4e92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4df9460 con 0x557ed4e91060 2023-10-08T09:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4deac20 con 0x557ed4e92800 2023-10-08T09:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.160+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4de8540 con 0x557ed4e919c0 2023-10-08T09:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.165+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 0x557ed4e92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.165+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 0x557ed4e92c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38212/0 (socket says 172.21.15.97:38212) 2023-10-08T09:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.165+0000 7fdc677fe700 1 -- 172.21.15.97:0/2570882924 learned_addr learned my addr 172.21.15.97:0/2570882924 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.167+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 0x557ed4e91480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4e92270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 0x7fdc540a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc66ffd700 1 -- 172.21.15.97:0/2570882924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 msgr2=0x557ed4e92270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4e92270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc66ffd700 1 -- 172.21.15.97:0/2570882924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 msgr2=0x557ed4e92c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 0x557ed4e92c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.169+0000 7fdc66ffd700 1 -- 172.21.15.97:0/2570882924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ed4c0ede0 con 0x557ed4e91060 2023-10-08T09:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.170+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 0x557ed4e91480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c50 tx=0x7fdc50009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36b4c2364fd440e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.171+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 0x7fdc540a3b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005950 tx=0x7fdc5800b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.175+0000 7fdc5dffb700 1 -- 172.21.15.97:0/2570882924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000fb40 con 0x557ed4e91060 2023-10-08T09:58:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.175+0000 7fdc5dffb700 1 -- 172.21.15.97:0/2570882924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5000fca0 con 0x557ed4e91060 2023-10-08T09:58:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.175+0000 7fdc5dffb700 1 -- 172.21.15.97:0/2570882924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500165d0 con 0x557ed4e91060 2023-10-08T09:58:01.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.177+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2570882924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 msgr2=0x557ed4e91480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.177+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 0x557ed4e91480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c50 tx=0x7fdc50009f60 comp rx=0 tx=0).stop 2023-10-08T09:58:01.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2570882924 shutdown_connections 2023-10-08T09:58:01.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e92800 0x557ed4e92c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e91060 0x557ed4e91480 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2570882924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4e92270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2570882924 >> 172.21.15.97:0/2570882924 conn(0x557ed4e05260 msgr2=0x557ed4e8be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2570882924 shutdown_connections 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.186+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2570882924 wait complete. 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 -- start start 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 Processor -- start 2023-10-08T09:58:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eabea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4de3e30 con 0x557ed4e92800 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4d80710 con 0x557ed4e91060 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4d803b0 con 0x557ed4e919c0 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.187+0000 7fdc79baeb80 10 client.0 _ll_register_callbacks cb 0x557ed4bdd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.188+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.188+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58252/0 (socket says 172.21.15.97:58252) 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.188+0000 7fdc677fe700 1 -- 172.21.15.97:0/1772730293 learned_addr learned my addr 172.21.15.97:0/1772730293 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.190+0000 7fdc677fe700 1 -- 172.21.15.97:0/1772730293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 msgr2=0x557ed4eac3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.190+0000 7fdc677fe700 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eac3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.190+0000 7fdc677fe700 1 -- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 msgr2=0x557ed4eabea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.190+0000 7fdc677fe700 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eabea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.190+0000 7fdc677fe700 1 -- 172.21.15.97:0/1772730293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc50009880 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.191+0000 7fdc677fe700 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009fd0 tx=0x7fdc600a3340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c905cd2d93aef657 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.191+0000 7fdc46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.192+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a3750 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.192+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc600a78a0 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.192+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ed4dabeb0 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.195+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc600aba50 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.195+0000 7fdc79baeb80 10 client.16134 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.195+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557ed4bf7c60 con 0x557ed4e92800 2023-10-08T09:58:01.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.195+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ed4dabeb0 con 0x557ed4e92800 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdc60019050 con 0x557ed4e92800 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 10 client.16134 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 20 client.16134 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4e9d660 0x557ed4e9fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 4 client.16134 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:01.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.196+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ed4ddce10 con 0x557ed4e9d660 2023-10-08T09:58:01.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.202+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4e9d660 0x557ed4e9fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.217+0000 7fdc46ffd700 10 client.16134 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:01.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.217+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4e9d660 0x557ed4e9fac0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f52512cad8179 server_cookie=63d2ad1d297d9005 in_seq=0 out_seq=0 2023-10-08T09:58:01.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.218+0000 7fdc46ffd700 1 -- 172.21.15.97:0/1772730293 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ed4ddce10 con 0x557ed4e9d660 2023-10-08T09:58:01.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.218+0000 7fdc46ffd700 10 client.16134 handle_command_reply: tid=0 2023-10-08T09:58:01.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.218+0000 7fdc79baeb80 1 client.16134 shutdown 2023-10-08T09:58:01.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.218+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 msgr2=0x557ed4eb33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009fd0 tx=0x7fdc600a3340 comp rx=0 tx=0).stop 2023-10-08T09:58:01.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 shutdown_connections 2023-10-08T09:58:01.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb33d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eabea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4e9d660 0x557ed4e9fac0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1772730293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.219+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 >> 172.21.15.97:0/1772730293 conn(0x557ed4e05260 msgr2=0x557ed4e8b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.229+0000 7fdc46ffd700 0 client.16134 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:01.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.229+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 shutdown_connections 2023-10-08T09:58:01.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.242+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1772730293 wait complete. 2023-10-08T09:58:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.248+0000 7fdc79baeb80 20 client.16134 trim_cache size 0 max 16384 2023-10-08T09:58:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.282+0000 7fdc79baeb80 1 Processor -- start 2023-10-08T09:58:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 -- start start 2023-10-08T09:58:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4e9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e92800 0x557ed4eab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4c64920 con 0x557ed4e919c0 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4e9a220 con 0x557ed4e91060 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4eaa940 con 0x557ed4e92800 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58262/0 (socket says 172.21.15.97:58262) 2023-10-08T09:58:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/1398242570 learned_addr learned my addr 172.21.15.97:0/1398242570 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.283+0000 7fdc677fe700 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e92800 0x557ed4eab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/1398242570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e92800 msgr2=0x557ed4eab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e92800 0x557ed4eab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/1398242570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 msgr2=0x557ed4e9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4e9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/1398242570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ed4c35250 con 0x557ed4e919c0 2023-10-08T09:58:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.284+0000 7fdc5e7fc700 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdc580115e0 tx=0x7fdc58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4ebc5679231331 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.287+0000 7fdc5effd700 1 -- 172.21.15.97:0/1398242570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004830 con 0x557ed4e919c0 2023-10-08T09:58:01.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.287+0000 7fdc5effd700 1 -- 172.21.15.97:0/1398242570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc58004e50 con 0x557ed4e919c0 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.287+0000 7fdc5effd700 1 -- 172.21.15.97:0/1398242570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5801d6f0 con 0x557ed4e919c0 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.296+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1398242570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 msgr2=0x557ed4eb3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdc580115e0 tx=0x7fdc58016000 comp rx=0 tx=0).stop 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1398242570 shutdown_connections 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e92800 0x557ed4eab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4e9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/1398242570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e919c0 0x557ed4eb3390 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1398242570 >> 172.21.15.97:0/1398242570 conn(0x557ed4e05260 msgr2=0x557ed4e98e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1398242570 shutdown_connections 2023-10-08T09:58:01.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 -- 172.21.15.97:0/1398242570 wait complete. 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.297+0000 7fdc79baeb80 1 -- start start 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 Processor -- start 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eb05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eb0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4eb1f00 con 0x557ed4e92800 2023-10-08T09:58:01.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4eb2070 con 0x557ed4e91060 2023-10-08T09:58:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed4eb2230 con 0x557ed4e919c0 2023-10-08T09:58:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.298+0000 7fdc79baeb80 10 client.0 _ll_register_callbacks cb 0x557ed4bdd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.299+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eb0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.299+0000 7fdc5e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eb0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:59964/0 (socket says 172.21.15.97:59964) 2023-10-08T09:58:01.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.299+0000 7fdc5e7fc700 1 -- 172.21.15.97:0/2888033369 learned_addr learned my addr 172.21.15.97:0/2888033369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 -- 172.21.15.97:0/2888033369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 msgr2=0x557ed4eb0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eb0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 -- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 msgr2=0x557ed4eb05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eb05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 -- 172.21.15.97:0/2888033369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5800bc30 con 0x557ed4e92800 2023-10-08T09:58:01.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.300+0000 7fdc677fe700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb19c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a3450 tx=0x7fdc600ab5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e785931246fe6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.305+0000 7fdc46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:01.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.305+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6009f740 con 0x557ed4e92800 2023-10-08T09:58:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.305+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc6009f8a0 con 0x557ed4e92800 2023-10-08T09:58:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.305+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc5000fca0 con 0x557ed4e92800 2023-10-08T09:58:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.305+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc600a78a0 con 0x557ed4e92800 2023-10-08T09:58:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 10 client.16140 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:01.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557ed4c0d2b0 con 0x557ed4e92800 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5000fca0 con 0x557ed4e92800 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdc60019070 con 0x557ed4e92800 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 20 client.16140 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4f1db60 0x557ed4f1ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:01.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 4 client.16140 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:58:01.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.306+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ed4ddcb30 con 0x557ed4f1db60 2023-10-08T09:58:01.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.311+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4f1db60 0x557ed4f1ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.317+0000 7fdc66ffd700 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4f1db60 0x557ed4f1ff50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e677ef3bfe1ef5f server_cookie=c6d7a98556196be5 in_seq=0 out_seq=0 2023-10-08T09:58:01.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.317+0000 7fdc46ffd700 10 client.16140 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:01.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.327+0000 7fdc46ffd700 1 -- 172.21.15.97:0/2888033369 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ed4ddcb30 con 0x557ed4f1db60 2023-10-08T09:58:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.327+0000 7fdc46ffd700 10 client.16140 handle_command_reply: tid=0 2023-10-08T09:58:01.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.327+0000 7fdc79baeb80 1 client.16140 shutdown 2023-10-08T09:58:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.329+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 msgr2=0x557ed4eb19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.329+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb19c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a3450 tx=0x7fdc600ab5c0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 shutdown_connections 2023-10-08T09:58:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed4e92800 0x557ed4eb19c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557ed4e91060 0x557ed4eb05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x557ed4f1db60 0x557ed4f1ff50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 --2- 172.21.15.97:0/2888033369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557ed4e919c0 0x557ed4eb0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 >> 172.21.15.97:0/2888033369 conn(0x557ed4e05260 msgr2=0x557ed4eab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc46ffd700 0 client.16140 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 shutdown_connections 2023-10-08T09:58:01.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.335+0000 7fdc79baeb80 1 -- 172.21.15.97:0/2888033369 wait complete. 2023-10-08T09:58:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.337+0000 7fdc79baeb80 20 client.16140 trim_cache size 0 max 16384 2023-10-08T09:58:01.349 INFO:teuthology.orchestra.run.smithi097.stdout:[] 2023-10-08T09:58:01.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.362+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 msgr2=0x7fdc540a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.362+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 0x7fdc540a3b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005950 tx=0x7fdc5800b510 comp rx=0 tx=0).stop 2023-10-08T09:58:01.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.362+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 msgr2=0x7fdc68195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:01.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.362+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 0x7fdc68195870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009750 tx=0x7fdc600079e0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 shutdown_connections 2023-10-08T09:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdc540a1680 0x7fdc540a3b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc681064d0 0x7fdc68195870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc68101270 0x7fdc6806f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 --2- 172.21.15.97:0/3495938138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc68103ba0 0x7fdc6806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:01.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 >> 172.21.15.97:0/3495938138 conn(0x7fdc680fac40 msgr2=0x7fdc68101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:01.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.366+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 shutdown_connections 2023-10-08T09:58:01.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:01.367+0000 7fdc6d9c5700 1 -- 172.21.15.97:0/3495938138 wait complete. 2023-10-08T09:58:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:01 smithi186 ceph-mon[104393]: pgmap v386: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 MiB/s wr, 11.38k op/s 2023-10-08T09:58:01.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:01 smithi097 ceph-mon[99491]: pgmap v386: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 MiB/s wr, 11.38k op/s 2023-10-08T09:58:01.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:01 smithi178 ceph-mon[103156]: pgmap v386: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 69 MiB/s wr, 11.38k op/s 2023-10-08T09:58:02.404 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:58:02.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:58:02.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 -- 172.21.15.97:0/2177795438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00ff4f0 msgr2=0x7f35f00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2177795438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00ff4f0 0x7f35f00ff910 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f35e00099c0 tx=0x7f35e0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 -- 172.21.15.97:0/2177795438 shutdown_connections 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2177795438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0101080 0x7f35f0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2177795438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00ff4f0 0x7f35f00ff910 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2177795438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 0x7f35f0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 -- 172.21.15.97:0/2177795438 >> 172.21.15.97:0/2177795438 conn(0x7f35f00faa40 msgr2=0x7f35f00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 -- 172.21.15.97:0/2177795438 shutdown_connections 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.655+0000 7f35f57a0700 1 -- 172.21.15.97:0/2177795438 wait complete. 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 Processor -- start 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 -- start start 2023-10-08T09:58:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f00ff4f0 0x7f35f01954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 0x7f35f01959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019cfd0 con 0x7f35f0101080 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019d110 con 0x7f35f01006e0 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f57a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f019d2a0 con 0x7f35f00ff4f0 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58282/0 (socket says 172.21.15.97:58282) 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 -- 172.21.15.97:0/2825285874 learned_addr learned my addr 172.21.15.97:0/2825285874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35e7fff700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 0x7f35f01959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35effff700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f00ff4f0 0x7f35f01954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 -- 172.21.15.97:0/2825285874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f00ff4f0 msgr2=0x7f35f01954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f00ff4f0 0x7f35f01954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 -- 172.21.15.97:0/2825285874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 msgr2=0x7f35f01959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 0x7f35f01959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 -- 172.21.15.97:0/2825285874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e00095f0 con 0x7f35f0101080 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f35e800ef30 tx=0x7f35e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.656+0000 7f35edffb700 1 -- 172.21.15.97:0/2825285874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8018070 con 0x7f35f0101080 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.657+0000 7f35edffb700 1 -- 172.21.15.97:0/2825285874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e800f040 con 0x7f35f0101080 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.657+0000 7f35edffb700 1 -- 172.21.15.97:0/2825285874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800cd00 con 0x7f35f0101080 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.657+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f019d520 con 0x7f35f0101080 2023-10-08T09:58:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.657+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f019d9e0 con 0x7f35f0101080 2023-10-08T09:58:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.659+0000 7f35edffb700 1 -- 172.21.15.97:0/2825285874 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f35e8003a00 con 0x7f35f0101080 2023-10-08T09:58:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.660+0000 7f35edffb700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 0x7f35dc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.660+0000 7f35edffb700 1 -- 172.21.15.97:0/2825285874 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f35e80ccb50 con 0x7f35f0101080 2023-10-08T09:58:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.661+0000 7f3602bebb80 1 Processor -- start 2023-10-08T09:58:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 -- start start 2023-10-08T09:58:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c8210 0x55e8a05c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a052d4e0 con 0x55e8a05c78b0 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a051eca0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.666+0000 7f3602bebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a051c5c0 con 0x55e8a05c8210 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.667+0000 7f35f4f9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.667+0000 7f35f4f9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38254/0 (socket says 172.21.15.97:38254) 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.667+0000 7f35f4f9f700 1 -- 172.21.15.97:0/273552130 learned_addr learned my addr 172.21.15.97:0/273552130 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 -- 172.21.15.97:0/273552130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c8210 msgr2=0x55e8a05c64a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c8210 0x55e8a05c64a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 -- 172.21.15.97:0/273552130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 msgr2=0x55e8a05c7cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05c7cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 -- 172.21.15.97:0/273552130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a0342e60 con 0x55e8a05c6aa0 2023-10-08T09:58:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.668+0000 7f35f4f9f700 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35e8096640 tx=0x7f35e8021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3232aa0e6fab4a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.674+0000 7f35e67fc700 1 -- 172.21.15.97:0/273552130 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800ce80 con 0x55e8a05c6aa0 2023-10-08T09:58:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.674+0000 7f35e67fc700 1 -- 172.21.15.97:0/273552130 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e801b320 con 0x55e8a05c6aa0 2023-10-08T09:58:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.675+0000 7f35e67fc700 1 -- 172.21.15.97:0/273552130 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e809a090 con 0x55e8a05c6aa0 2023-10-08T09:58:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.675+0000 7f35effff700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 0x7f35dc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.676+0000 7f3602bebb80 1 -- 172.21.15.97:0/273552130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 msgr2=0x55e8a05c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.676+0000 7f35effff700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 0x7f35dc0a39c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f35e0003730 tx=0x7f35e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.676+0000 7f3602bebb80 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35e8096640 tx=0x7f35e8021000 comp rx=0 tx=0).stop 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.678+0000 7f3602bebb80 1 -- 172.21.15.97:0/273552130 shutdown_connections 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.678+0000 7f3602bebb80 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a05c6f50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.678+0000 7f3602bebb80 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.678+0000 7f3602bebb80 1 --2- 172.21.15.97:0/273552130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c8210 0x55e8a05c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.678+0000 7f3602bebb80 1 -- 172.21.15.97:0/273552130 >> 172.21.15.97:0/273552130 conn(0x55e8a05392e0 msgr2=0x55e8a05c0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.680+0000 7f3602bebb80 1 -- 172.21.15.97:0/273552130 shutdown_connections 2023-10-08T09:58:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.681+0000 7f3602bebb80 1 -- 172.21.15.97:0/273552130 wait complete. 2023-10-08T09:58:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.682+0000 7f3602bebb80 1 -- start start 2023-10-08T09:58:02.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.682+0000 7f3602bebb80 1 Processor -- start 2023-10-08T09:58:02.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c78b0 0x55e8a05e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a0517eb0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a04b4790 con 0x55e8a05c78b0 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a04b4430 con 0x55e8a05edeb0 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.694+0000 7f3602bebb80 10 client.0 _ll_register_callbacks cb 0x55e8a0311880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58288/0 (socket says 172.21.15.97:58288) 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 -- 172.21.15.97:0/770786024 learned_addr learned my addr 172.21.15.97:0/770786024 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 -- 172.21.15.97:0/770786024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 msgr2=0x55e8a05f02a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05f02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 -- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c78b0 msgr2=0x55e8a05e8e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c78b0 0x55e8a05e8e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 -- 172.21.15.97:0/770786024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f019d9e0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.696+0000 7f35effff700 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35e0015000 tx=0x7f35e0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7423aabbd437ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.705+0000 7f35cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.705+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0029070 con 0x55e8a05c6aa0 2023-10-08T09:58:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.705+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f35e0003dc0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.716+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8a04dff30 con 0x55e8a05c6aa0 2023-10-08T09:58:02.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.717+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e000f460 con 0x55e8a05c6aa0 2023-10-08T09:58:02.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.720+0000 7f3602bebb80 10 client.16149 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:02.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.720+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e8a032bce0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8a04dff30 con 0x55e8a05c6aa0 2023-10-08T09:58:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f35e0035070 con 0x55e8a05c6aa0 2023-10-08T09:58:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 20 client.16149 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05f4b10 0x55e8a05f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 4 client.16149 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.723+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8a0510e90 con 0x55e8a05f4b10 2023-10-08T09:58:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.730+0000 7f35e7fff700 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05f4b10 0x55e8a05f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.732+0000 7f35cf7fe700 10 client.16149 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:02.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.732+0000 7f35e7fff700 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05f4b10 0x55e8a05f6f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a8737dabcbf836 server_cookie=3a9c81f887c07ac0 in_seq=0 out_seq=0 2023-10-08T09:58:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.735+0000 7f35cf7fe700 1 -- 172.21.15.97:0/770786024 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8a0510e90 con 0x55e8a05f4b10 2023-10-08T09:58:02.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.735+0000 7f35cf7fe700 10 client.16149 handle_command_reply: tid=0 2023-10-08T09:58:02.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.736+0000 7f3602bebb80 1 client.16149 shutdown 2023-10-08T09:58:02.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.736+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 msgr2=0x55e8a05e8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.736+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35e0015000 tx=0x7f35e0009310 comp rx=0 tx=0).stop 2023-10-08T09:58:02.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 shutdown_connections 2023-10-08T09:58:02.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05f4b10 0x55e8a05f6f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a05e8940 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 --2- 172.21.15.97:0/770786024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c78b0 0x55e8a05e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.742+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 >> 172.21.15.97:0/770786024 conn(0x55e8a05392e0 msgr2=0x55e8a05c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.744+0000 7f35cf7fe700 0 client.16149 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:02.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.744+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 shutdown_connections 2023-10-08T09:58:02.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.745+0000 7f3602bebb80 1 -- 172.21.15.97:0/770786024 wait complete. 2023-10-08T09:58:02.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.747+0000 7f3602bebb80 20 client.16149 trim_cache size 0 max 16384 2023-10-08T09:58:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.793+0000 7f3602bebb80 1 Processor -- start 2023-10-08T09:58:02.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 -- start start 2023-10-08T09:58:02.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a03989a0 con 0x55e8a05c78b0 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a05f1fc0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.794+0000 7f3602bebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a05e9a20 con 0x55e8a05edeb0 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38262/0 (socket says 172.21.15.97:38262) 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 -- 172.21.15.97:0/922288627 learned_addr learned my addr 172.21.15.97:0/922288627 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 -- 172.21.15.97:0/922288627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 msgr2=0x55e8a05d2dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05d2dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 -- 172.21.15.97:0/922288627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 msgr2=0x55e8a05d2880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05d2880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.795+0000 7f35effff700 1 -- 172.21.15.97:0/922288627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a03692d0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.796+0000 7f35effff700 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35e000bf40 tx=0x7f35e0005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945c85c13b331229 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.797+0000 7f35e6ffd700 1 -- 172.21.15.97:0/922288627 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0029070 con 0x55e8a05c6aa0 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.797+0000 7f35e6ffd700 1 -- 172.21.15.97:0/922288627 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e001e040 con 0x55e8a05c6aa0 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.797+0000 7f35e6ffd700 1 -- 172.21.15.97:0/922288627 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0022bd0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.798+0000 7f3602bebb80 1 -- 172.21.15.97:0/922288627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 msgr2=0x55e8a0604710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.798+0000 7f3602bebb80 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35e000bf40 tx=0x7f35e0005960 comp rx=0 tx=0).stop 2023-10-08T09:58:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.800+0000 7f3602bebb80 1 -- 172.21.15.97:0/922288627 shutdown_connections 2023-10-08T09:58:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.800+0000 7f3602bebb80 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05edeb0 0x55e8a05d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.800+0000 7f3602bebb80 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05c6aa0 0x55e8a0604710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.800+0000 7f3602bebb80 1 --2- 172.21.15.97:0/922288627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c78b0 0x55e8a05d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.800+0000 7f3602bebb80 1 -- 172.21.15.97:0/922288627 >> 172.21.15.97:0/922288627 conn(0x55e8a05392e0 msgr2=0x55e8a05c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.809+0000 7f3602bebb80 1 -- 172.21.15.97:0/922288627 shutdown_connections 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.809+0000 7f3602bebb80 1 -- 172.21.15.97:0/922288627 wait complete. 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.809+0000 7f3602bebb80 1 -- start start 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 Processor -- start 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a06027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05edeb0 0x55e8a0603bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a06040f0 con 0x55e8a05c6aa0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a0604260 con 0x55e8a05edeb0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a0604420 con 0x55e8a05c78b0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.810+0000 7f3602bebb80 10 client.0 _ll_register_callbacks cb 0x7f35e000bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.812+0000 7f35f4f9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05edeb0 0x55e8a0603bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.812+0000 7f35e7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.812+0000 7f35e7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60000/0 (socket says 172.21.15.97:60000) 2023-10-08T09:58:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.812+0000 7f35e7fff700 1 -- 172.21.15.97:0/1251285715 learned_addr learned my addr 172.21.15.97:0/1251285715 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 -- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05edeb0 msgr2=0x55e8a0603bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05edeb0 0x55e8a0603bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 -- 172.21.15.97:0/1251285715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 msgr2=0x55e8a06027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a06027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 -- 172.21.15.97:0/1251285715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8a032bce0 con 0x55e8a05c78b0 2023-10-08T09:58:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35e7fff700 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ba0 tx=0x7f35d8019b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7b30760369bc3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.813+0000 7f35cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-08T09:58:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.823+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8004250 con 0x55e8a05c78b0 2023-10-08T09:58:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.823+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f35d8016070 con 0x55e8a05c78b0 2023-10-08T09:58:02.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.823+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35e801b320 con 0x55e8a05c78b0 2023-10-08T09:58:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.830+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d80043b0 con 0x55e8a05c78b0 2023-10-08T09:58:02.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.833+0000 7f3602bebb80 10 client.25522 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:02.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.833+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e8a0341330 con 0x55e8a05c78b0 2023-10-08T09:58:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e801b320 con 0x55e8a05c78b0 2023-10-08T09:58:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f35d8004a50 con 0x55e8a05c78b0 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 10 client.25522 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 20 client.25522 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05cf750 0x55e8a05cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 4 client.25522 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:58:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.836+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e8a0510bb0 con 0x55e8a05cf750 2023-10-08T09:58:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.837+0000 7f35effff700 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05cf750 0x55e8a05cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:02.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.838+0000 7f35effff700 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05cf750 0x55e8a05cfb60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3b08eddea3c400 server_cookie=fba1cad9079581f9 in_seq=0 out_seq=0 2023-10-08T09:58:02.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.839+0000 7f35cf7fe700 10 client.25522 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:02.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.853+0000 7f35cf7fe700 1 -- 172.21.15.97:0/1251285715 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e8a0510bb0 con 0x55e8a05cf750 2023-10-08T09:58:02.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.853+0000 7f35cf7fe700 10 client.25522 handle_command_reply: tid=0 2023-10-08T09:58:02.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.853+0000 7f3602bebb80 1 client.25522 shutdown 2023-10-08T09:58:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.856+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 msgr2=0x55e8a0602ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.856+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ba0 tx=0x7f35d8019b60 comp rx=0 tx=0).stop 2023-10-08T09:58:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 shutdown_connections 2023-10-08T09:58:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e8a05edeb0 0x55e8a0603bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a05c6aa0 0x55e8a06027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55e8a05cf750 0x55e8a05cfb60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 --2- 172.21.15.97:0/1251285715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8a05c78b0 0x55e8a0602ce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 >> 172.21.15.97:0/1251285715 conn(0x55e8a05392e0 msgr2=0x55e8a05d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.861+0000 7f35cf7fe700 0 client.25522 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:02.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.865+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 shutdown_connections 2023-10-08T09:58:02.880 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:58:02.929 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2023-10-08T09:58:02.929 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "344ff07b-09ad-4c45-b96b-9a40b46dbe0a", 2023-10-08T09:58:02.929 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2023-10-08T09:58:02.929 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.876+0000 7f3602bebb80 1 -- 172.21.15.97:0/1251285715 wait complete. 2023-10-08T09:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.877+0000 7f3602bebb80 20 client.25522 trim_cache size 0 max 16384 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.885+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 msgr2=0x7f35dc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.885+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 0x7f35dc0a39c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f35e0003730 tx=0x7f35e0000bc0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.885+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 msgr2=0x7f35f019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.885+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f35e800ef30 tx=0x7f35e800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 shutdown_connections 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f35dc0a1500 0x7f35dc0a39c0 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f35e0003730 tx=0x7f35e0000bc0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0101080 0x7f35f019ca00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f00ff4f0 0x7f35f01954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 --2- 172.21.15.97:0/2825285874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35f01006e0 0x7f35f01959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.888+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 >> 172.21.15.97:0/2825285874 conn(0x7f35f00faa40 msgr2=0x7f35f0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.894+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 shutdown_connections 2023-10-08T09:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:02.897+0000 7f35f57a0700 1 -- 172.21.15.97:0/2825285874 wait complete. 2023-10-08T09:58:02.944 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:58:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.230+0000 7f0446f48700 1 -- 172.21.15.97:0/1640626549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 msgr2=0x7f044010d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.230+0000 7f0446f48700 1 --2- 172.21.15.97:0/1640626549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f044010d620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0438009f90 tx=0x7f043800b330 comp rx=0 tx=0).stop 2023-10-08T09:58:03.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.232+0000 7f0446f48700 1 -- 172.21.15.97:0/1640626549 shutdown_connections 2023-10-08T09:58:03.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.232+0000 7f0446f48700 1 --2- 172.21.15.97:0/1640626549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044010db60 0x7f044010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.232+0000 7f0446f48700 1 --2- 172.21.15.97:0/1640626549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0440108900 0x7f044010acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.232+0000 7f0446f48700 1 --2- 172.21.15.97:0/1640626549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f044010d620 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.232+0000 7f0446f48700 1 -- 172.21.15.97:0/1640626549 >> 172.21.15.97:0/1640626549 conn(0x7f044006d790 msgr2=0x7f044006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.246+0000 7f0446f48700 1 -- 172.21.15.97:0/1640626549 shutdown_connections 2023-10-08T09:58:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.254+0000 7f0446f48700 1 -- 172.21.15.97:0/1640626549 wait complete. 2023-10-08T09:58:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.255+0000 7f0446f48700 1 Processor -- start 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.255+0000 7f0446f48700 1 -- start start 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.255+0000 7f0446f48700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.255+0000 7f0446f48700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044010db60 0x7f0440117580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.256+0000 7f0446f48700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04401185b0 0x7f0440118a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.256+0000 7f0446f48700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0440118fa0 con 0x7f044010b230 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.256+0000 7f0446f48700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0440119110 con 0x7f044010db60 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.256+0000 7f0446f48700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04401192d0 con 0x7f04401185b0 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58310/0 (socket says 172.21.15.97:58310) 2023-10-08T09:58:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 -- 172.21.15.97:0/1154249206 learned_addr learned my addr 172.21.15.97:0/1154249206 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 -- 172.21.15.97:0/1154249206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04401185b0 msgr2=0x7f0440118a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04401185b0 0x7f0440118a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 -- 172.21.15.97:0/1154249206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044010db60 msgr2=0x7f0440117580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044010db60 0x7f0440117580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.257+0000 7f0444ce4700 1 -- 172.21.15.97:0/1154249206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009c40 con 0x7f044010b230 2023-10-08T09:58:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.258+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f043000d900 tx=0x7f043000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.258+0000 7f043dffb700 1 -- 172.21.15.97:0/1154249206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430009930 con 0x7f044010b230 2023-10-08T09:58:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.259+0000 7f043dffb700 1 -- 172.21.15.97:0/1154249206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f043000b5c0 con 0x7f044010b230 2023-10-08T09:58:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.259+0000 7f043dffb700 1 -- 172.21.15.97:0/1154249206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000f3c0 con 0x7f044010b230 2023-10-08T09:58:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.259+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0440078a50 con 0x7f044010b230 2023-10-08T09:58:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.259+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0440078f10 con 0x7f044010b230 2023-10-08T09:58:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.264+0000 7f0453131b80 1 Processor -- start 2023-10-08T09:58:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 -- start start 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f4111480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4112c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4079460 con 0x5607f41119c0 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f406ac20 con 0x5607f4111060 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.267+0000 7f0453131b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4068540 con 0x5607f4112800 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58316/0 (socket says 172.21.15.97:58316) 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 -- 172.21.15.97:0/601589804 learned_addr learned my addr 172.21.15.97:0/601589804 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 -- 172.21.15.97:0/601589804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 msgr2=0x5607f4112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 -- 172.21.15.97:0/601589804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 msgr2=0x5607f4111480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f4111480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 -- 172.21.15.97:0/601589804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607f3e8ede0 con 0x5607f41119c0 2023-10-08T09:58:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.268+0000 7f0444ce4700 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0440078c40 tx=0x7f0430009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59ee921d13f97e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.270+0000 7f043dffb700 1 -- 172.21.15.97:0/1154249206 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0430005520 con 0x7f044010b230 2023-10-08T09:58:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.271+0000 7f043dffb700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 0x7f04280a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.271+0000 7f043dffb700 1 -- 172.21.15.97:0/1154249206 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f04300ce470 con 0x7f044010b230 2023-10-08T09:58:03.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.276+0000 7f04454e5700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 0x7f04280a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.276+0000 7f04454e5700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 0x7f04280a3ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0434005fd0 tx=0x7f0434009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.277+0000 7f0426ffd700 1 -- 172.21.15.97:0/601589804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300d4030 con 0x5607f41119c0 2023-10-08T09:58:03.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.277+0000 7f0426ffd700 1 -- 172.21.15.97:0/601589804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0430097f10 con 0x5607f41119c0 2023-10-08T09:58:03.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.277+0000 7f0426ffd700 1 -- 172.21.15.97:0/601589804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300cfe30 con 0x5607f41119c0 2023-10-08T09:58:03.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 -- 172.21.15.97:0/601589804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 msgr2=0x5607f4112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0440078c40 tx=0x7f0430009ea0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 -- 172.21.15.97:0/601589804 shutdown_connections 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f4111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 --2- 172.21.15.97:0/601589804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f4112270 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.278+0000 7f0453131b80 1 -- 172.21.15.97:0/601589804 >> 172.21.15.97:0/601589804 conn(0x5607f4085260 msgr2=0x5607f410be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.280+0000 7f0453131b80 1 -- 172.21.15.97:0/601589804 shutdown_connections 2023-10-08T09:58:03.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.281+0000 7f0453131b80 1 -- 172.21.15.97:0/601589804 wait complete. 2023-10-08T09:58:03.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.281+0000 7f0453131b80 1 -- start start 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.281+0000 7f0453131b80 1 Processor -- start 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4063e30 con 0x5607f41119c0 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4000710 con 0x5607f4111060 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f40003b0 con 0x5607f4112800 2023-10-08T09:58:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.282+0000 7f0453131b80 10 client.0 _ll_register_callbacks cb 0x5607f3e5d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:03.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.283+0000 7f043ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58324/0 (socket says 172.21.15.97:58324) 2023-10-08T09:58:03.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 -- 172.21.15.97:0/792439396 learned_addr learned my addr 172.21.15.97:0/792439396 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f0444ce4700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 -- 172.21.15.97:0/792439396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 msgr2=0x5607f4122350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 -- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 msgr2=0x5607f411add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.284+0000 7f043ffff700 1 -- 172.21.15.97:0/792439396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0440078f10 con 0x5607f41119c0 2023-10-08T09:58:03.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.285+0000 7f043ffff700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0438004c60 tx=0x7f0438004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23038be0efc547a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f041ffff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438021070 con 0x5607f41119c0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f04380075f0 con 0x5607f41119c0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607f402beb0 con 0x5607f41119c0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043800e040 con 0x5607f41119c0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f0453131b80 10 client.16161 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.298+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5607f3e77c60 con 0x5607f41119c0 2023-10-08T09:58:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.299+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607f402beb0 con 0x5607f41119c0 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.299+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0438024020 con 0x5607f41119c0 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 20 client.16161 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f4126cd0 0x5607f41290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 4 client.16161 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.300+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607f405ce10 con 0x5607f4126cd0 2023-10-08T09:58:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.301+0000 7f0444ce4700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f4126cd0 0x5607f41290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.322+0000 7f041ffff700 10 client.16161 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:03.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.322+0000 7f0444ce4700 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f4126cd0 0x5607f41290f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1f02070b03ed4f server_cookie=f3a261d785055ad6 in_seq=0 out_seq=0 2023-10-08T09:58:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.327+0000 7f041ffff700 1 -- 172.21.15.97:0/792439396 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607f405ce10 con 0x5607f4126cd0 2023-10-08T09:58:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.327+0000 7f041ffff700 10 client.16161 handle_command_reply: tid=0 2023-10-08T09:58:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.329+0000 7f0453131b80 1 client.16161 shutdown 2023-10-08T09:58:03.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 msgr2=0x5607f411b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0438004c60 tx=0x7f0438004c90 comp rx=0 tx=0).stop 2023-10-08T09:58:03.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 shutdown_connections 2023-10-08T09:58:03.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f4126cd0 0x5607f41290f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 --2- 172.21.15.97:0/792439396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f41119c0 0x5607f411b310 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0438004c60 tx=0x7f0438004c90 comp rx=0 tx=0).stop 2023-10-08T09:58:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 >> 172.21.15.97:0/792439396 conn(0x5607f4085260 msgr2=0x5607f410b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.338+0000 7f041ffff700 0 client.16161 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:03.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.350+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 shutdown_connections 2023-10-08T09:58:03.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.350+0000 7f0453131b80 1 -- 172.21.15.97:0/792439396 wait complete. 2023-10-08T09:58:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.362+0000 7f0453131b80 20 client.16161 trim_cache size 0 max 16384 2023-10-08T09:58:03.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.399+0000 7f0453131b80 1 Processor -- start 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.399+0000 7f0453131b80 1 -- start start 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f3ee4920 con 0x5607f4107f30 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4124060 con 0x5607f4111060 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f0453131b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f4111e10 con 0x5607f4112800 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f04454e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f04454e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58330/0 (socket says 172.21.15.97:58330) 2023-10-08T09:58:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.400+0000 7f04454e5700 1 -- 172.21.15.97:0/3838869752 learned_addr learned my addr 172.21.15.97:0/3838869752 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 -- 172.21.15.97:0/3838869752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 msgr2=0x5607f4122310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 -- 172.21.15.97:0/3838869752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 msgr2=0x5607f411a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 -- 172.21.15.97:0/3838869752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607f3eb5250 con 0x5607f4107f30 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04454e5700 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f043400ea60 tx=0x7f0434011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b036266ca55e69b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04277fe700 1 -- 172.21.15.97:0/3838869752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434010840 con 0x5607f4107f30 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04277fe700 1 -- 172.21.15.97:0/3838869752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0434010e60 con 0x5607f4107f30 2023-10-08T09:58:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.401+0000 7f04277fe700 1 -- 172.21.15.97:0/3838869752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043403e430 con 0x5607f4107f30 2023-10-08T09:58:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.402+0000 7f0453131b80 1 -- 172.21.15.97:0/3838869752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 msgr2=0x5607f4118d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.402+0000 7f0453131b80 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f043400ea60 tx=0x7f0434011000 comp rx=0 tx=0).stop 2023-10-08T09:58:03.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.407+0000 7f0453131b80 1 -- 172.21.15.97:0/3838869752 shutdown_connections 2023-10-08T09:58:03.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.407+0000 7f0453131b80 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4118d00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.407+0000 7f0453131b80 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4111060 0x5607f411a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.407+0000 7f0453131b80 1 --2- 172.21.15.97:0/3838869752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4112800 0x5607f4122310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.407+0000 7f0453131b80 1 -- 172.21.15.97:0/3838869752 >> 172.21.15.97:0/3838869752 conn(0x5607f4085260 msgr2=0x5607f410bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.411+0000 7f0453131b80 1 -- 172.21.15.97:0/3838869752 shutdown_connections 2023-10-08T09:58:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.417+0000 7f0453131b80 1 -- 172.21.15.97:0/3838869752 wait complete. 2023-10-08T09:58:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.418+0000 7f0453131b80 1 -- start start 2023-10-08T09:58:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.418+0000 7f0453131b80 1 Processor -- start 2023-10-08T09:58:03.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4111060 0x5607f411ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4112800 0x5607f411f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f411f810 con 0x5607f4107f30 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f417aaf0 con 0x5607f4112800 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607f417ac60 con 0x5607f4111060 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.425+0000 7f0453131b80 10 client.0 _ll_register_callbacks cb 0x7f04340091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.426+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.426+0000 7f0444ce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58334/0 (socket says 172.21.15.97:58334) 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.426+0000 7f0444ce4700 1 -- 172.21.15.97:0/1198105471 learned_addr learned my addr 172.21.15.97:0/1198105471 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:03.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.427+0000 7f0444ce4700 1 -- 172.21.15.97:0/1198105471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4111060 msgr2=0x5607f411ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.427+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4111060 0x5607f411ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.427+0000 7f0444ce4700 1 -- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4112800 msgr2=0x5607f411f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.427+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4112800 0x5607f411f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.427+0000 7f0444ce4700 1 -- 172.21.15.97:0/1198105471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043400e740 con 0x5607f4107f30 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f0444ce4700 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f04300cbcd0 tx=0x7f043009bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8690d485034cec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f041ffff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300d4040 con 0x5607f4107f30 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f04300d5070 con 0x5607f4107f30 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0430097f10 con 0x5607f4107f30 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043009b600 con 0x5607f4107f30 2023-10-08T09:58:03.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f0453131b80 10 client.16167 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.429+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5607f3e8d2b0 con 0x5607f4107f30 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0430097f10 con 0x5607f4107f30 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0430009bc0 con 0x5607f4107f30 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 20 client.16167 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f417c3b0 0x5607f417e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 4 client.16167 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.430+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5607f405cb30 con 0x5607f417c3b0 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.431+0000 7f043ffff700 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f417c3b0 0x5607f417e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.432+0000 7f041ffff700 10 client.16167 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.432+0000 7f043ffff700 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f417c3b0 0x5607f417e810 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3d5af84349e66a server_cookie=ab77d4dde293df3f in_seq=0 out_seq=0 2023-10-08T09:58:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.432+0000 7f041ffff700 1 -- 172.21.15.97:0/1198105471 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5607f405cb30 con 0x5607f417c3b0 2023-10-08T09:58:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.432+0000 7f041ffff700 10 client.16167 handle_command_reply: tid=0 2023-10-08T09:58:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.432+0000 7f0453131b80 1 client.16167 shutdown 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.436+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 msgr2=0x5607f4122190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.436+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f04300cbcd0 tx=0x7f043009bdb0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 shutdown_connections 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607f4112800 0x5607f411f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607f4107f30 0x5607f4122190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5607f417c3b0 0x5607f417e810 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 --2- 172.21.15.97:0/1198105471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f4111060 0x5607f411ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.444+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 >> 172.21.15.97:0/1198105471 conn(0x5607f4085260 msgr2=0x5607f4128ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.445+0000 7f041ffff700 0 client.16167 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:03.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.446+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 shutdown_connections 2023-10-08T09:58:03.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.447+0000 7f0453131b80 1 -- 172.21.15.97:0/1198105471 wait complete. 2023-10-08T09:58:03.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.452+0000 7f0453131b80 20 client.16167 trim_cache size 0 max 16384 2023-10-08T09:58:03.463 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:58:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "scrub active (112 inodes in the stack)", 2023-10-08T09:58:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2023-10-08T09:58:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "344ff07b-09ad-4c45-b96b-9a40b46dbe0a": { 2023-10-08T09:58:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2023-10-08T09:58:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "344ff07b-09ad-4c45-b96b-9a40b46dbe0a", 2023-10-08T09:58:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2023-10-08T09:58:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:58:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:58:03.464 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:58:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.486+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 msgr2=0x7f04280a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.486+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 0x7f04280a3ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0434005fd0 tx=0x7f0434009310 comp rx=0 tx=0).stop 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.486+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 msgr2=0x7f0440117040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.486+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f043000d900 tx=0x7f043000dcc0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 shutdown_connections 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f04280a1600 0x7f04280a3ac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04401185b0 0x7f0440118a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044010b230 0x7f0440117040 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 --2- 172.21.15.97:0/1154249206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f044010db60 0x7f0440117580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.487+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 >> 172.21.15.97:0/1154249206 conn(0x7f044006d790 msgr2=0x7f0440108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.497+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 shutdown_connections 2023-10-08T09:58:03.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:03.497+0000 7f0446f48700 1 -- 172.21.15.97:0/1154249206 wait complete. 2023-10-08T09:58:03.530 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: pgmap v387: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 12.67k op/s 2023-10-08T09:58:03.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: scrub queued for path: / 2023-10-08T09:58:03.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:03.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:58:03.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:58:03.531 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:03 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T09:58:03.539 INFO:tasks.cephfs.filesystem:scrub status for tag:344ff07b-09ad-4c45-b96b-9a40b46dbe0a - {'path': '/', 'tag': '344ff07b-09ad-4c45-b96b-9a40b46dbe0a', 'options': 'recursive,force'} 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: pgmap v387: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 12.67k op/s 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: scrub queued for path: / 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:58:03.544 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:03 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: pgmap v387: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 12.67k op/s 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: scrub queued for path: / 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:58:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:03 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T09:58:05.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:05 smithi178 ceph-mon[103156]: pgmap v388: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 68 MiB/s wr, 11.48k op/s 2023-10-08T09:58:05.710 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:05 smithi097 ceph-mon[99491]: pgmap v388: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 68 MiB/s wr, 11.48k op/s 2023-10-08T09:58:05.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:05 smithi186 ceph-mon[104393]: pgmap v388: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 68 MiB/s wr, 11.48k op/s 2023-10-08T09:58:07.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:07 smithi097 ceph-mon[99491]: pgmap v389: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 65 MiB/s wr, 10.88k op/s 2023-10-08T09:58:07.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:07 smithi178 ceph-mon[103156]: pgmap v389: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 65 MiB/s wr, 10.88k op/s 2023-10-08T09:58:07.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:07 smithi186 ceph-mon[104393]: pgmap v389: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 65 MiB/s wr, 10.88k op/s 2023-10-08T09:58:08.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:08 smithi097 ceph-mon[99491]: pgmap v390: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 62 MiB/s wr, 10.40k op/s 2023-10-08T09:58:08.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:08 smithi178 ceph-mon[103156]: pgmap v390: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 62 MiB/s wr, 10.40k op/s 2023-10-08T09:58:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:08 smithi186 ceph-mon[104393]: pgmap v390: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 62 MiB/s wr, 10.40k op/s 2023-10-08T09:58:11.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:11 smithi186 ceph-mon[104393]: pgmap v391: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 10.18k op/s 2023-10-08T09:58:11.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:11 smithi097 ceph-mon[99491]: pgmap v391: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 10.18k op/s 2023-10-08T09:58:11.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:11 smithi178 ceph-mon[103156]: pgmap v391: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 10.18k op/s 2023-10-08T09:58:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.655+0000 7f2cf002a700 1 -- 172.21.15.97:0/3088004711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 msgr2=0x7f2ce80ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.655+0000 7f2cf002a700 1 --2- 172.21.15.97:0/3088004711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce80ffba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009990 tx=0x7f2cd8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.661+0000 7f2cf002a700 1 -- 172.21.15.97:0/3088004711 shutdown_connections 2023-10-08T09:58:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.661+0000 7f2cf002a700 1 --2- 172.21.15.97:0/3088004711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce8104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.661+0000 7f2cf002a700 1 --2- 172.21.15.97:0/3088004711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce80fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.661+0000 7f2cf002a700 1 --2- 172.21.15.97:0/3088004711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce80ffba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.661+0000 7f2cf002a700 1 -- 172.21.15.97:0/3088004711 >> 172.21.15.97:0/3088004711 conn(0x7f2ce80f9ef0 msgr2=0x7f2ce80fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.676+0000 7f2cf002a700 1 -- 172.21.15.97:0/3088004711 shutdown_connections 2023-10-08T09:58:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.680+0000 7f2cf002a700 1 -- 172.21.15.97:0/3088004711 wait complete. 2023-10-08T09:58:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.681+0000 7f2cf002a700 1 Processor -- start 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 -- start start 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce806f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80726a0 con 0x7f2ce80ff720 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8072810 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cf002a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce819c380 con 0x7f2ce81000e0 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2ceddc6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60044/0 (socket says 172.21.15.97:60044) 2023-10-08T09:58:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2cee5c7700 1 -- 172.21.15.97:0/704631261 learned_addr learned my addr 172.21.15.97:0/704631261 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.682+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce806f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 -- 172.21.15.97:0/704631261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 msgr2=0x7f2ce819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 -- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 msgr2=0x7f2ce806f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce806f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 -- 172.21.15.97:0/704631261 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.683+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce806f120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9f0 tx=0x7f2ce400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.695+0000 7f2cdeffd700 1 -- 172.21.15.97:0/704631261 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40096d0 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.695+0000 7f2cdeffd700 1 -- 172.21.15.97:0/704631261 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce400cbd0 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.695+0000 7f2cdeffd700 1 -- 172.21.15.97:0/704631261 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018680 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:13 smithi097 ceph-mon[99491]: pgmap v392: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 59 MiB/s wr, 9.93k op/s 2023-10-08T09:58:13.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:13 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:58:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.699+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce819c5b0 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.699+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce819ca70 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:13 smithi178 ceph-mon[103156]: pgmap v392: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 59 MiB/s wr, 9.93k op/s 2023-10-08T09:58:13.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:13 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:58:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.713+0000 7f2cdeffd700 1 -- 172.21.15.97:0/704631261 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2ce4010430 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.714+0000 7f2cdeffd700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 0x7f2cd40a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.714+0000 7f2cdeffd700 1 -- 172.21.15.97:0/704631261 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2ce4014070 con 0x7f2ce80fe6b0 2023-10-08T09:58:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.714+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 0x7f2cd40a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.715+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 0x7f2cd40a3a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b7b0 tx=0x7f2cd8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:13 smithi186 ceph-mon[104393]: pgmap v392: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 59 MiB/s wr, 9.93k op/s 2023-10-08T09:58:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:13 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:58:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.923+0000 7f2cfc213b80 1 Processor -- start 2023-10-08T09:58:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 -- start start 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ecdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15ece730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15e35400 con 0x55ce15ecd7a0 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15e26bc0 con 0x55ce15ece2b0 2023-10-08T09:58:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.925+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15e244e0 con 0x55ce15ecce40 2023-10-08T09:58:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.927+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15ece730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15ece730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38324/0 (socket says 172.21.15.97:38324) 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2cee5c7700 1 -- 172.21.15.97:0/538662528 learned_addr learned my addr 172.21.15.97:0/538662528 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60052/0 (socket says 172.21.15.97:60052) 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 -- 172.21.15.97:0/538662528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 msgr2=0x55ce15ece730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15ece730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 -- 172.21.15.97:0/538662528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 msgr2=0x55ce15ecdcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ecdcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.928+0000 7f2ceddc6700 1 -- 172.21.15.97:0/538662528 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce15c4ad80 con 0x55ce15ecce40 2023-10-08T09:58:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.938+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c5b0 tx=0x7f2ce409a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4caa75646359b1ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.939+0000 7f2cd3fff700 1 -- 172.21.15.97:0/538662528 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce409e160 con 0x55ce15ecce40 2023-10-08T09:58:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.939+0000 7f2cd3fff700 1 -- 172.21.15.97:0/538662528 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce409a920 con 0x55ce15ecce40 2023-10-08T09:58:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.939+0000 7f2cd3fff700 1 -- 172.21.15.97:0/538662528 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce409a480 con 0x55ce15ecce40 2023-10-08T09:58:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.941+0000 7f2cfc213b80 1 -- 172.21.15.97:0/538662528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 msgr2=0x55ce15ecd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.941+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c5b0 tx=0x7f2ce409a070 comp rx=0 tx=0).stop 2023-10-08T09:58:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 -- 172.21.15.97:0/538662528 shutdown_connections 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15ece730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ecd260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/538662528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ecdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 -- 172.21.15.97:0/538662528 >> 172.21.15.97:0/538662528 conn(0x55ce15e41200 msgr2=0x55ce15ec78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 -- 172.21.15.97:0/538662528 shutdown_connections 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.943+0000 7f2cfc213b80 1 -- 172.21.15.97:0/538662528 wait complete. 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 -- start start 2023-10-08T09:58:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 Processor -- start 2023-10-08T09:58:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15e1fdd0 con 0x55ce15ecd7a0 2023-10-08T09:58:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15dbc6b0 con 0x55ce15ece2b0 2023-10-08T09:58:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15dbc350 con 0x55ce15ecce40 2023-10-08T09:58:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.944+0000 7f2cfc213b80 10 client.0 _ll_register_callbacks cb 0x55ce15c197a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58358/0 (socket says 172.21.15.97:58358) 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 -- 172.21.15.97:0/1100844075 learned_addr learned my addr 172.21.15.97:0/1100844075 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 -- 172.21.15.97:0/1100844075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 msgr2=0x55ce15ed6940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed6940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 -- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 msgr2=0x55ce15edde20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edde20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.945+0000 7f2ced5c5700 1 -- 172.21.15.97:0/1100844075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce819ca70 con 0x55ce15ecd7a0 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.946+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009990 tx=0x7f2cd8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb6718e3cdb2d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.946+0000 7f2cd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.947+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802b070 con 0x55ce15ecd7a0 2023-10-08T09:58:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.947+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2cd80056c0 con 0x55ce15ecd7a0 2023-10-08T09:58:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.951+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce15de7e50 con 0x55ce15ecd7a0 2023-10-08T09:58:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.951+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd8004a70 con 0x55ce15ecd7a0 2023-10-08T09:58:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.953+0000 7f2cfc213b80 10 client.16176 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.953+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ce15c33c00 con 0x55ce15ecd7a0 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce15de7e50 con 0x55ce15ecd7a0 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2cd800f460 con 0x55ce15ecd7a0 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 20 client.16176 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15ee2760 0x55ce15ee4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 4 client.16176 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.954+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce15e18db0 con 0x55ce15ee2760 2023-10-08T09:58:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:13.974+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15ee2760 0x55ce15ee4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.002+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15ee2760 0x55ce15ee4b80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9991bd25f7ed880d server_cookie=1ea3eb8ebceca56b in_seq=0 out_seq=0 2023-10-08T09:58:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.003+0000 7f2cd17fa700 10 client.16176 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.005+0000 7f2cd17fa700 1 -- 172.21.15.97:0/1100844075 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce15e18db0 con 0x55ce15ee2760 2023-10-08T09:58:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.005+0000 7f2cd17fa700 10 client.16176 handle_command_reply: tid=0 2023-10-08T09:58:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.005+0000 7f2cfc213b80 1 client.16176 shutdown 2023-10-08T09:58:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.012+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 msgr2=0x55ce15ed6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.012+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009990 tx=0x7f2cd8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 shutdown_connections 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15ee2760 0x55ce15ee4b80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/1100844075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15ed6e80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 >> 172.21.15.97:0/1100844075 conn(0x55ce15e41200 msgr2=0x55ce15ec7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.020+0000 7f2cd17fa700 0 client.16176 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.026+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 shutdown_connections 2023-10-08T09:58:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.030+0000 7f2cfc213b80 1 -- 172.21.15.97:0/1100844075 wait complete. 2023-10-08T09:58:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.031+0000 7f2cfc213b80 20 client.16176 trim_cache size 0 max 16384 2023-10-08T09:58:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.060+0000 7f2cfc213b80 1 Processor -- start 2023-10-08T09:58:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 -- start start 2023-10-08T09:58:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 0x55ce15eddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 0x55ce15ed1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15ca08c0 con 0x55ce15ece2b0 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15ec96e0 con 0x55ce15ecd7a0 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15e18db0 con 0x55ce15ecce40 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 0x55ce15ed1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 0x55ce15ed1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58364/0 (socket says 172.21.15.97:58364) 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2ceddc6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.066+0000 7f2cee5c7700 1 -- 172.21.15.97:0/3003027858 learned_addr learned my addr 172.21.15.97:0/3003027858 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 0x55ce15eddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 -- 172.21.15.97:0/3003027858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 msgr2=0x55ce15ed12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 -- 172.21.15.97:0/3003027858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 msgr2=0x55ce15ed1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 0x55ce15ed1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 -- 172.21.15.97:0/3003027858 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce15c711f0 con 0x55ce15ecd7a0 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.067+0000 7f2ced5c5700 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 0x55ce15eddde0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8000c00 tx=0x7f2cd80056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6b76b712d2a69d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.068+0000 7f2cdcff9700 1 -- 172.21.15.97:0/3003027858 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802b040 con 0x55ce15ecd7a0 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.068+0000 7f2cdcff9700 1 -- 172.21.15.97:0/3003027858 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd800fe90 con 0x55ce15ecd7a0 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.068+0000 7f2cdcff9700 1 -- 172.21.15.97:0/3003027858 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803ab10 con 0x55ce15ecd7a0 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.068+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3003027858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 msgr2=0x55ce15eddde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.068+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 0x55ce15eddde0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8000c00 tx=0x7f2cd80056c0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3003027858 shutdown_connections 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ece2b0 0x55ce15ed1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15ed12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3003027858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ecd7a0 0x55ce15eddde0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3003027858 >> 172.21.15.97:0/3003027858 conn(0x55ce15e41200 msgr2=0x55ce15ed5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3003027858 shutdown_connections 2023-10-08T09:58:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.072+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3003027858 wait complete. 2023-10-08T09:58:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 -- start start 2023-10-08T09:58:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 Processor -- start 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15edb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15edba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15edcee0 con 0x55ce15ecd7a0 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15edd050 con 0x55ce15ece2b0 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce15edd210 con 0x55ce15ecce40 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.073+0000 7f2cfc213b80 10 client.0 _ll_register_callbacks cb 0x7f2cd8005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.074+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.074+0000 7f2cee5c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38344/0 (socket says 172.21.15.97:38344) 2023-10-08T09:58:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.074+0000 7f2cee5c7700 1 -- 172.21.15.97:0/3837414636 learned_addr learned my addr 172.21.15.97:0/3837414636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 -- 172.21.15.97:0/3837414636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 msgr2=0x55ce15edb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15edb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 -- 172.21.15.97:0/3837414636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 msgr2=0x55ce15edba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15edba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 -- 172.21.15.97:0/3837414636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce15c33c00 con 0x55ce15ece2b0 2023-10-08T09:58:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cee5c7700 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007e50 tx=0x7f2ce0007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1696032b4da6a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.075+0000 7f2cd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004db0 con 0x55ce15ece2b0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mon.2 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2ce0015950 con 0x55ce15ece2b0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ce409a920 con 0x55ce15ece2b0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce0015b10 con 0x55ce15ece2b0 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cfc213b80 10 client.25577 fetch_fsmap learned FSMap version 26 2023-10-08T09:58:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.076+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ce15c49250 con 0x55ce15ece2b0 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.077+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mon.2 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce409a920 con 0x55ce15ece2b0 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.077+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mon.2 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2ce0005020 con 0x55ce15ece2b0 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 26 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 20 client.25577 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15f36f20 0x55ce15f39340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 4 client.25577 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:58:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.078+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ce15e18ad0 con 0x55ce15f36f20 2023-10-08T09:58:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.079+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15f36f20 0x55ce15f39340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.081+0000 7f2ceddc6700 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15f36f20 0x55ce15f39340 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da05a7a7ea933ba server_cookie=a5f27f050f1afddb in_seq=0 out_seq=0 2023-10-08T09:58:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:14.082+0000 7f2cd17fa700 10 client.25577 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: pgmap v393: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 9.60k op/s 2023-10-08T09:58:14.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:14 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: pgmap v393: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 9.60k op/s 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:14.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:14 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: pgmap v393: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 9.60k op/s 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:15.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:14 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:16.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:58:16.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:58:16.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:16.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:16 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:16.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:58:16.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:58:16.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:16.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:16 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:58:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:58:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:58:16.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:16 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:17 smithi097 ceph-mon[99491]: pgmap v394: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 10.15k op/s 2023-10-08T09:58:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:17 smithi178 ceph-mon[103156]: pgmap v394: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 10.15k op/s 2023-10-08T09:58:17.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:17 smithi186 ceph-mon[104393]: pgmap v394: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 10.15k op/s 2023-10-08T09:58:19.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:19 smithi097 ceph-mon[99491]: pgmap v395: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 9.16k op/s 2023-10-08T09:58:19.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:19 smithi178 ceph-mon[103156]: pgmap v395: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 9.16k op/s 2023-10-08T09:58:19.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:19 smithi186 ceph-mon[104393]: pgmap v395: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 9.16k op/s 2023-10-08T09:58:20.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:20 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:20.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:20 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:20.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:20 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T09:58:21.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:21 smithi097 ceph-mon[99491]: pgmap v396: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 9.36k op/s 2023-10-08T09:58:21.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:21 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:21.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:21 smithi178 ceph-mon[103156]: pgmap v396: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 9.36k op/s 2023-10-08T09:58:21.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:21 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:21.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:21 smithi186 ceph-mon[104393]: pgmap v396: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 MiB/s wr, 9.36k op/s 2023-10-08T09:58:21.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:21 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:58:22.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:22 smithi097 ceph-mon[99491]: pgmap v397: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 9.29k op/s 2023-10-08T09:58:22.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:22 smithi178 ceph-mon[103156]: pgmap v397: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 9.29k op/s 2023-10-08T09:58:22.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:22 smithi186 ceph-mon[104393]: pgmap v397: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 MiB/s wr, 9.29k op/s 2023-10-08T09:58:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cd17fa700 1 -- 172.21.15.97:0/3837414636 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ce15e18ad0 con 0x55ce15f36f20 2023-10-08T09:58:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cd17fa700 10 client.25577 handle_command_reply: tid=0 2023-10-08T09:58:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 client.25577 shutdown 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 msgr2=0x55ce15edc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007e50 tx=0x7f2ce0007f30 comp rx=0 tx=0).stop 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 shutdown_connections 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce15ece2b0 0x55ce15edc9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55ce15f36f20 0x55ce15f39340 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ce15ecce40 0x55ce15edb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 --2- 172.21.15.97:0/3837414636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce15ecd7a0 0x55ce15edba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cd17fa700 0 client.25577 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 >> 172.21.15.97:0/3837414636 conn(0x55ce15e41200 msgr2=0x55ce15ed12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 shutdown_connections 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 1 -- 172.21.15.97:0/3837414636 wait complete. 2023-10-08T09:58:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.892+0000 7f2cfc213b80 20 client.25577 trim_cache size 0 max 16384 2023-10-08T09:58:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T09:58:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T09:58:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T09:58:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 msgr2=0x7f2cd40a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 0x7f2cd40a3a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b7b0 tx=0x7f2cd8005fd0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 msgr2=0x7f2ce806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce806f120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9f0 tx=0x7f2ce400ed00 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 shutdown_connections 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2cd40a15d0 0x7f2cd40a3a90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ce81000e0 0x7f2ce819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce80fe6b0 0x7f2ce806f120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 --2- 172.21.15.97:0/704631261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce80ff720 0x7f2ce806f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 >> 172.21.15.97:0/704631261 conn(0x7f2ce80f9ef0 msgr2=0x7f2ce8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 shutdown_connections 2023-10-08T09:58:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:23.893+0000 7f2cf002a700 1 -- 172.21.15.97:0/704631261 wait complete. 2023-10-08T09:58:25.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:25 smithi097 ceph-mon[99491]: pgmap v398: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 MiB/s wr, 8.91k op/s 2023-10-08T09:58:25.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:25 smithi178 ceph-mon[103156]: pgmap v398: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 MiB/s wr, 8.91k op/s 2023-10-08T09:58:25.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:25 smithi186 ceph-mon[104393]: pgmap v398: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 48 MiB/s wr, 8.91k op/s 2023-10-08T09:58:27.524 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:27 smithi186 ceph-mon[104393]: pgmap v399: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 MiB/s wr, 9.60k op/s 2023-10-08T09:58:27.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:27 smithi097 ceph-mon[99491]: pgmap v399: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 MiB/s wr, 9.60k op/s 2023-10-08T09:58:27.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:27 smithi178 ceph-mon[103156]: pgmap v399: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 MiB/s wr, 9.60k op/s 2023-10-08T09:58:29.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:29 smithi097 ceph-mon[99491]: pgmap v400: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 7.75k op/s 2023-10-08T09:58:29.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:29 smithi178 ceph-mon[103156]: pgmap v400: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 7.75k op/s 2023-10-08T09:58:29.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:29 smithi186 ceph-mon[104393]: pgmap v400: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 MiB/s wr, 7.75k op/s 2023-10-08T09:58:31.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:31 smithi178 ceph-mon[103156]: pgmap v401: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 7.45k op/s 2023-10-08T09:58:31.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:31 smithi097 ceph-mon[99491]: pgmap v401: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 7.45k op/s 2023-10-08T09:58:31.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:31 smithi186 ceph-mon[104393]: pgmap v401: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 7.45k op/s 2023-10-08T09:58:33.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:58:33.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:33 smithi178 ceph-mon[103156]: pgmap v402: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 7.63k op/s 2023-10-08T09:58:33.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:33 smithi097 ceph-mon[99491]: pgmap v402: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 7.63k op/s 2023-10-08T09:58:33.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:33 smithi186 ceph-mon[104393]: pgmap v402: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 7.63k op/s 2023-10-08T09:58:34.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.226+0000 7f17154bd700 1 -- 172.21.15.97:0/1020343011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 msgr2=0x7f171010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.226+0000 7f17154bd700 1 --2- 172.21.15.97:0/1020343011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f171010cfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f170001a9a0 tx=0x7f170001acb0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.228+0000 7f17154bd700 1 -- 172.21.15.97:0/1020343011 shutdown_connections 2023-10-08T09:58:34.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.228+0000 7f17154bd700 1 --2- 172.21.15.97:0/1020343011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f171010cfb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.228+0000 7f17154bd700 1 --2- 172.21.15.97:0/1020343011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17101077f0 0x7f1710109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.228+0000 7f17154bd700 1 --2- 172.21.15.97:0/1020343011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f171010a160 0x7f171010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.228+0000 7f17154bd700 1 -- 172.21.15.97:0/1020343011 >> 172.21.15.97:0/1020343011 conn(0x7f171006d790 msgr2=0x7f171006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.235+0000 7f17154bd700 1 -- 172.21.15.97:0/1020343011 shutdown_connections 2023-10-08T09:58:34.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.246+0000 7f17154bd700 1 -- 172.21.15.97:0/1020343011 wait complete. 2023-10-08T09:58:34.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 Processor -- start 2023-10-08T09:58:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 -- start start 2023-10-08T09:58:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17101077f0 0x7f171019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f171010a160 0x7f171019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101c6890 con 0x7f17101077f0 2023-10-08T09:58:34.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101c6a00 con 0x7f171010a160 2023-10-08T09:58:34.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.247+0000 7f17154bd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101c6bc0 con 0x7f171010cb30 2023-10-08T09:58:34.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.252+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.252+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60084/0 (socket says 172.21.15.97:60084) 2023-10-08T09:58:34.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.252+0000 7f170f7fe700 1 -- 172.21.15.97:0/2811677510 learned_addr learned my addr 172.21.15.97:0/2811677510 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:34.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.252+0000 7f170e7fc700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f171010a160 0x7f171019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 -- 172.21.15.97:0/2811677510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f171010a160 msgr2=0x7f171019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f171010a160 0x7f171019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 -- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17101077f0 msgr2=0x7f171019cd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17101077f0 0x7f171019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 -- 172.21.15.97:0/2811677510 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170001a600 con 0x7f171010cb30 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f170f7fe700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17000059d0 tx=0x7f170001cfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.253+0000 7f16f7fff700 1 -- 172.21.15.97:0/2811677510 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700004030 con 0x7f171010cb30 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.254+0000 7f16f7fff700 1 -- 172.21.15.97:0/2811677510 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1700004190 con 0x7f171010cb30 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.254+0000 7f16f7fff700 1 -- 172.21.15.97:0/2811677510 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170001f6e0 con 0x7f171010cb30 2023-10-08T09:58:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.254+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17101c6e40 con 0x7f171010cb30 2023-10-08T09:58:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.254+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17101c71b0 con 0x7f171010cb30 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.259+0000 7f17216a6b80 1 Processor -- start 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 -- start start 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db140bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd9c0 0x55db140be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 0x55db140bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14025460 con 0x55db140bd060 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14016c20 con 0x55db140bd9c0 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.261+0000 7f17216a6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14014540 con 0x55db140be800 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f170e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 0x55db140bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f170e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 0x55db140bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60092/0 (socket says 172.21.15.97:60092) 2023-10-08T09:58:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f170e7fc700 1 -- 172.21.15.97:0/2872759637 learned_addr learned my addr 172.21.15.97:0/2872759637 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f170e7fc700 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db140bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f16f7fff700 1 -- 172.21.15.97:0/2811677510 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f170000f480 con 0x7f171010cb30 2023-10-08T09:58:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.263+0000 7f170e7fc700 1 -- 172.21.15.97:0/2872759637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 msgr2=0x55db140bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f170e7fc700 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 0x55db140bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f170e7fc700 1 -- 172.21.15.97:0/2872759637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd9c0 msgr2=0x55db140be270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:58:34.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f170e7fc700 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd9c0 0x55db140be270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f170e7fc700 1 -- 172.21.15.97:0/2872759637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db13e3ade0 con 0x55db140bd060 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f16f7fff700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 0x7f16f80a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f170e7fc700 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db140bd480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1704006ea0 tx=0x7f1704009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddea89ff2a6ee21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f16f7fff700 1 -- 172.21.15.97:0/2811677510 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f17000ce6a0 con 0x7f171010cb30 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f16f57fa700 1 -- 172.21.15.97:0/2872759637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17040097b0 con 0x55db140bd060 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f16f57fa700 1 -- 172.21.15.97:0/2872759637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1704015e60 con 0x55db140bd060 2023-10-08T09:58:34.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.264+0000 7f16f57fa700 1 -- 172.21.15.97:0/2872759637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704018910 con 0x55db140bd060 2023-10-08T09:58:34.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.265+0000 7f17216a6b80 1 -- 172.21.15.97:0/2872759637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 msgr2=0x55db140bd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.265+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db140bd480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1704006ea0 tx=0x7f1704009190 comp rx=0 tx=0).stop 2023-10-08T09:58:34.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.266+0000 7f17216a6b80 1 -- 172.21.15.97:0/2872759637 shutdown_connections 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.266+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140be800 0x55db140bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.266+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db140bd480 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.266+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2872759637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd9c0 0x55db140be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.266+0000 7f17216a6b80 1 -- 172.21.15.97:0/2872759637 >> 172.21.15.97:0/2872759637 conn(0x55db14031260 msgr2=0x55db140b7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.267+0000 7f170effd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 0x7f16f80a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.268+0000 7f17216a6b80 1 -- 172.21.15.97:0/2872759637 shutdown_connections 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.268+0000 7f17216a6b80 1 -- 172.21.15.97:0/2872759637 wait complete. 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.268+0000 7f17216a6b80 1 -- start start 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 Processor -- start 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140bd060 0x55db140c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140cc4e0 0x55db140ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1400fe30 con 0x55db140be800 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db13fac710 con 0x55db140cc4e0 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db13fac3b0 con 0x55db140bd060 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f17216a6b80 10 client.0 _ll_register_callbacks cb 0x55db13e09800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f170e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f170e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58396/0 (socket says 172.21.15.97:58396) 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f170e7fc700 1 -- 172.21.15.97:0/3897756798 learned_addr learned my addr 172.21.15.97:0/3897756798 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f170f7fe700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140cc4e0 0x55db140ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.269+0000 7f170effd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 0x7f16f80a3b10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55db140b9a30 tx=0x7f17080091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 -- 172.21.15.97:0/3897756798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140bd060 msgr2=0x55db140c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140bd060 0x55db140c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 -- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140cc4e0 msgr2=0x55db140ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140cc4e0 0x55db140ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 -- 172.21.15.97:0/3897756798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1704015490 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1704018010 tx=0x7f1704017d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b658dd662efe3b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.270+0000 7f16f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704017a60 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1704018690 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db13fd7eb0 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1704017e10 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f17216a6b80 10 client.16185 fetch_fsmap learned FSMap version 28 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.271+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55db13e23c60 con 0x55db140be800 2023-10-08T09:58:34.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.272+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db13fd7eb0 con 0x55db140be800 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.272+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f17040218f0 con 0x55db140be800 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 20 client.16185 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db140d3020 0x55db140d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 4 client.16185 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.273+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db14008e10 con 0x55db140d3020 2023-10-08T09:58:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.280+0000 7f170effd700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db140d3020 0x55db140d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.281+0000 7f170effd700 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db140d3020 0x55db140d54e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a448f78853c4db04 server_cookie=4babab16b82d0aa in_seq=0 out_seq=0 2023-10-08T09:58:34.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.281+0000 7f16f2ff5700 10 client.16185 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.283+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3897756798 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55db14008e10 con 0x55db140d3020 2023-10-08T09:58:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.283+0000 7f16f2ff5700 10 client.16185 handle_command_reply: tid=0 2023-10-08T09:58:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.283+0000 7f17216a6b80 1 client.16185 shutdown 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 msgr2=0x55db140c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1704018010 tx=0x7f1704017d70 comp rx=0 tx=0).stop 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 shutdown_connections 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140cc4e0 0x55db140ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db140d3020 0x55db140d54e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140bd060 0x55db140c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3897756798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140be800 0x55db140c7500 secure :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1704018010 tx=0x7f1704017d70 comp rx=0 tx=0).stop 2023-10-08T09:58:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 >> 172.21.15.97:0/3897756798 conn(0x55db14031260 msgr2=0x55db140b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.284+0000 7f16f2ff5700 0 client.16185 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.286+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 shutdown_connections 2023-10-08T09:58:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.286+0000 7f17216a6b80 1 -- 172.21.15.97:0/3897756798 wait complete. 2023-10-08T09:58:34.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.289+0000 7f17216a6b80 20 client.16185 trim_cache size 0 max 16384 2023-10-08T09:58:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 Processor -- start 2023-10-08T09:58:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 -- start start 2023-10-08T09:58:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db140e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140b3f30 0x55db140c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db13e90920 con 0x55db140b3f30 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db140b97e0 con 0x55db140bd060 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f17216a6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db140cfa60 con 0x55db140cc4e0 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.323+0000 7f170effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38378/0 (socket says 172.21.15.97:38378) 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 -- 172.21.15.97:0/2364349192 learned_addr learned my addr 172.21.15.97:0/2364349192 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 -- 172.21.15.97:0/2364349192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 msgr2=0x55db140e2d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db140e2d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 -- 172.21.15.97:0/2364349192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140b3f30 msgr2=0x55db140c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140b3f30 0x55db140c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 -- 172.21.15.97:0/2364349192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db13e61250 con 0x55db140bd060 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.324+0000 7f170effd700 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55db140bde00 tx=0x7f170800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510efe411a721b88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.327+0000 7f16f5ffb700 1 -- 172.21.15.97:0/2364349192 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708021070 con 0x55db140bd060 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.327+0000 7f16f5ffb700 1 -- 172.21.15.97:0/2364349192 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1708021e80 con 0x55db140bd060 2023-10-08T09:58:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.327+0000 7f16f5ffb700 1 -- 172.21.15.97:0/2364349192 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170800ee90 con 0x55db140bd060 2023-10-08T09:58:34.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 -- 172.21.15.97:0/2364349192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 msgr2=0x55db140c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55db140bde00 tx=0x7f170800f000 comp rx=0 tx=0).stop 2023-10-08T09:58:34.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 -- 172.21.15.97:0/2364349192 shutdown_connections 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140b3f30 0x55db140c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140bd060 0x55db140c4f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 --2- 172.21.15.97:0/2364349192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db140e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 -- 172.21.15.97:0/2364349192 >> 172.21.15.97:0/2364349192 conn(0x55db14031260 msgr2=0x55db140d9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 -- 172.21.15.97:0/2364349192 shutdown_connections 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.335+0000 7f17216a6b80 1 -- 172.21.15.97:0/2364349192 wait complete. 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.336+0000 7f17216a6b80 1 -- start start 2023-10-08T09:58:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.336+0000 7f17216a6b80 1 Processor -- start 2023-10-08T09:58:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140b3f30 0x55db14121820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db14123d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db141263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14126af0 con 0x55db140bd060 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14126c60 con 0x55db140b3f30 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db14126e20 con 0x55db140cc4e0 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.340+0000 7f17216a6b80 10 client.0 _ll_register_callbacks cb 0x7f1704009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.341+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db141263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.341+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db141263b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60114/0 (socket says 172.21.15.97:60114) 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.341+0000 7f170f7fe700 1 -- 172.21.15.97:0/3203005867 learned_addr learned my addr 172.21.15.97:0/3203005867 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.343+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db14123d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170effd700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140b3f30 0x55db14121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170e7fc700 1 -- 172.21.15.97:0/3203005867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 msgr2=0x55db141263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db141263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170e7fc700 1 -- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140b3f30 msgr2=0x55db14121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:34.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140b3f30 0x55db14121820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.345+0000 7f170e7fc700 1 -- 172.21.15.97:0/3203005867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170800c8f0 con 0x55db140bd060 2023-10-08T09:58:34.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.347+0000 7f170e7fc700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db14123d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55db140b8de0 tx=0x7f1704009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e065e072ff51728e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:34.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.347+0000 7f16f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:34.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.350+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704009ed0 con 0x55db140bd060 2023-10-08T09:58:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.350+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1704000e40 con 0x55db140bd060 2023-10-08T09:58:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.350+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db14008b30 con 0x55db140bd060 2023-10-08T09:58:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.353+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1704006d30 con 0x55db140bd060 2023-10-08T09:58:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.377+0000 7f17216a6b80 10 client.16188 fetch_fsmap learned FSMap version 28 2023-10-08T09:58:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.377+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55db13e392b0 con 0x55db140bd060 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.378+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db14008b30 con 0x55db140bd060 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.378+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f170403e050 con 0x55db140bd060 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 20 client.16188 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db14128590 0x55db1412a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 4 client.16188 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:58:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.380+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db13e5bf70 con 0x55db14128590 2023-10-08T09:58:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.381+0000 7f170effd700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db14128590 0x55db1412a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:34.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.397+0000 7f170effd700 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db14128590 0x55db1412a9b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4361760ad994ca3f server_cookie=c02cd7cdff682129 in_seq=0 out_seq=0 2023-10-08T09:58:34.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.397+0000 7f16f2ff5700 10 client.16188 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.400+0000 7f16f2ff5700 1 -- 172.21.15.97:0/3203005867 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55db13e5bf70 con 0x55db14128590 2023-10-08T09:58:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.400+0000 7f16f2ff5700 10 client.16188 handle_command_reply: tid=0 2023-10-08T09:58:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.402+0000 7f17216a6b80 1 client.16188 shutdown 2023-10-08T09:58:34.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.415+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 msgr2=0x55db14123d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.415+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db14123d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55db140b8de0 tx=0x7f1704009190 comp rx=0 tx=0).stop 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 shutdown_connections 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55db14128590 0x55db1412a9b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db140cc4e0 0x55db141263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db140b3f30 0x55db14121820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 --2- 172.21.15.97:0/3203005867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db140bd060 0x55db14123d70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 >> 172.21.15.97:0/3203005867 conn(0x55db14031260 msgr2=0x55db140c4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.417+0000 7f16f2ff5700 0 client.16188 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:34.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.419+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 shutdown_connections 2023-10-08T09:58:34.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.419+0000 7f17216a6b80 1 -- 172.21.15.97:0/3203005867 wait complete. 2023-10-08T09:58:34.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.426+0000 7f17216a6b80 20 client.16188 trim_cache size 0 max 16384 2023-10-08T09:58:34.438 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T09:58:34.438 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "scrub active (25 inodes in the stack)", 2023-10-08T09:58:34.439 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2023-10-08T09:58:34.439 INFO:teuthology.orchestra.run.smithi097.stdout: "344ff07b-09ad-4c45-b96b-9a40b46dbe0a": { 2023-10-08T09:58:34.439 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2023-10-08T09:58:34.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "344ff07b-09ad-4c45-b96b-9a40b46dbe0a", 2023-10-08T09:58:34.440 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive,force" 2023-10-08T09:58:34.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:58:34.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T09:58:34.440 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T09:58:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 msgr2=0x7f16f80a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 0x7f16f80a3b10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55db140b9a30 tx=0x7f17080091f0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 msgr2=0x7f17101c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17000059d0 tx=0x7f170001cfb0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 shutdown_connections 2023-10-08T09:58:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f16f80a1650 0x7f16f80a3b10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f171010cb30 0x7f17101c62c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17101077f0 0x7f171019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 --2- 172.21.15.97:0/2811677510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f171010a160 0x7f171019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:34.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.445+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 >> 172.21.15.97:0/2811677510 conn(0x7f171006d790 msgr2=0x7f171010a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.458+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 shutdown_connections 2023-10-08T09:58:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:58:34.458+0000 7f17154bd700 1 -- 172.21.15.97:0/2811677510 wait complete. 2023-10-08T09:58:34.497 INFO:tasks.cephfs.filesystem:scrub status for tag:344ff07b-09ad-4c45-b96b-9a40b46dbe0a - {'path': '/', 'tag': '344ff07b-09ad-4c45-b96b-9a40b46dbe0a', 'options': 'recursive,force'} 2023-10-08T09:58:35.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:35 smithi097 ceph-mon[99491]: pgmap v403: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 6.83k op/s 2023-10-08T09:58:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:35 smithi178 ceph-mon[103156]: pgmap v403: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 6.83k op/s 2023-10-08T09:58:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:35 smithi186 ceph-mon[104393]: pgmap v403: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 6.83k op/s 2023-10-08T09:58:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.091+0000 7f85aa6b3700 1 -- 172.21.15.97:0/389495798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 msgr2=0x7f85a410a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.091+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/389495798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a410a250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b0a0 tx=0x7f859c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 -- 172.21.15.97:0/389495798 shutdown_connections 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/389495798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a4110f30 0x7f85a4113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/389495798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4072630 0x7f85a4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/389495798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a410a250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 -- 172.21.15.97:0/389495798 >> 172.21.15.97:0/389495798 conn(0x7f85a406d6e0 msgr2=0x7f85a406fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 -- 172.21.15.97:0/389495798 shutdown_connections 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.095+0000 7f85aa6b3700 1 -- 172.21.15.97:0/389495798 wait complete. 2023-10-08T09:58:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.096+0000 7f85aa6b3700 1 Processor -- start 2023-10-08T09:58:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.098+0000 7f85aa6b3700 1 -- start start 2023-10-08T09:58:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 0x7f85a41a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41a90a0 con 0x7f85a4110f30 2023-10-08T09:58:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41a91e0 con 0x7f85a41a66e0 2023-10-08T09:58:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.099+0000 7f85aa6b3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41a9370 con 0x7f85a4109dd0 2023-10-08T09:58:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.101+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.101+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58426/0 (socket says 172.21.15.97:58426) 2023-10-08T09:58:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.101+0000 7f85a37fe700 1 -- 172.21.15.97:0/3198039455 learned_addr learned my addr 172.21.15.97:0/3198039455 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.101+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 0x7f85a41a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a3fff700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a8c50700 1 -- 172.21.15.97:0/3198039455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 msgr2=0x7f85a41a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a41a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a8c50700 1 -- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 msgr2=0x7f85a41a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a8c50700 1 -- 172.21.15.97:0/3198039455 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c009d00 con 0x7f85a41a66e0 2023-10-08T09:58:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.102+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:58:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.104+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 0x7f85a41a8ad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f859400b3f0 tx=0x7f859400b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.109+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594011830 con 0x7f85a41a66e0 2023-10-08T09:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.109+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8594011e50 con 0x7f85a41a66e0 2023-10-08T09:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.109+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594010db0 con 0x7f85a41a66e0 2023-10-08T09:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.109+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a41a9650 con 0x7f85a41a66e0 2023-10-08T09:58:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.109+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a410add0 con 0x7f85a41a66e0 2023-10-08T09:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.114+0000 7f85b689cb80 1 Processor -- start 2023-10-08T09:58:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.116+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8594011990 con 0x7f85a41a66e0 2023-10-08T09:58:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.116+0000 7f85b689cb80 1 -- start start 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.116+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fd6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.116+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fd7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85b689cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621f2d400 con 0x55f621fd67b0 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85b689cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621f1ebc0 con 0x55f621fd72c0 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85b689cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621f1c4e0 con 0x55f621fd5e50 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85a17fa700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 0x7f858c0a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f85940cc8d0 con 0x7f85a41a66e0 2023-10-08T09:58:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58428/0 (socket says 172.21.15.97:58428) 2023-10-08T09:58:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.117+0000 7f85a3fff700 1 -- 172.21.15.97:0/2851518017 learned_addr learned my addr 172.21.15.97:0/2851518017 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a37fe700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fd6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a37fe700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fd7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 -- 172.21.15.97:0/2851518017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 msgr2=0x55f621fd6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fd6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 -- 172.21.15.97:0/2851518017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 msgr2=0x55f621fd7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fd7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 -- 172.21.15.97:0/2851518017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f621d42d80 con 0x55f621fd67b0 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a37fe700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fd7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:58:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.118+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8598007810 tx=0x7f8598007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8494deea8144c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.120+0000 7f858a7fc700 1 -- 172.21.15.97:0/2851518017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980125f0 con 0x55f621fd67b0 2023-10-08T09:58:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.120+0000 7f858a7fc700 1 -- 172.21.15.97:0/2851518017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8598012c10 con 0x55f621fd67b0 2023-10-08T09:58:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.120+0000 7f858a7fc700 1 -- 172.21.15.97:0/2851518017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598013dc0 con 0x55f621fd67b0 2023-10-08T09:58:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.121+0000 7f85b689cb80 1 -- 172.21.15.97:0/2851518017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 msgr2=0x55f621fd6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.121+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8598007810 tx=0x7f8598007b20 comp rx=0 tx=0).stop 2023-10-08T09:58:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.129+0000 7f85b689cb80 1 -- 172.21.15.97:0/2851518017 shutdown_connections 2023-10-08T09:58:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.129+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fd7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.129+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fd6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.129+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2851518017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fd6ce0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.129+0000 7f85b689cb80 1 -- 172.21.15.97:0/2851518017 >> 172.21.15.97:0/2851518017 conn(0x55f621f39200 msgr2=0x55f621fbf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.133+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 0x7f858c0a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.134+0000 7f85a8c50700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 0x7f858c0a3ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8594096330 tx=0x7f8594010850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.136+0000 7f85b689cb80 1 -- 172.21.15.97:0/2851518017 shutdown_connections 2023-10-08T09:58:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.139+0000 7f85b689cb80 1 -- 172.21.15.97:0/2851518017 wait complete. 2023-10-08T09:58:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.139+0000 7f85b689cb80 1 -- start start 2023-10-08T09:58:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.139+0000 7f85b689cb80 1 Processor -- start 2023-10-08T09:58:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd5e50 0x55f621fdf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd67b0 0x55f621fdfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621f17dd0 con 0x55f621fd72c0 2023-10-08T09:58:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621eb46b0 con 0x55f621fd5e50 2023-10-08T09:58:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621eb4350 con 0x55f621fd67b0 2023-10-08T09:58:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.142+0000 7f85b689cb80 10 client.0 _ll_register_callbacks cb 0x55f621d117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58438/0 (socket says 172.21.15.97:58438) 2023-10-08T09:58:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 -- 172.21.15.97:0/3106539762 learned_addr learned my addr 172.21.15.97:0/3106539762 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a3fff700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd67b0 0x55f621fdfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd5e50 0x55f621fdf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 -- 172.21.15.97:0/3106539762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd67b0 msgr2=0x55f621fdfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd67b0 0x55f621fdfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 -- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd5e50 msgr2=0x55f621fdf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd5e50 0x55f621fdf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.143+0000 7f85a37fe700 1 -- 172.21.15.97:0/3106539762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859800e040 con 0x55f621fd72c0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f859c000f80 tx=0x7f859c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b50f8fa14d9a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85837fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00e030 con 0x55f621fd72c0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f859c00f070 con 0x55f621fd72c0 2023-10-08T09:58:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f621edfe50 con 0x55f621fd72c0 2023-10-08T09:58:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f859c01fc30 con 0x55f621fd72c0 2023-10-08T09:58:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85b689cb80 10 client.16197 fetch_fsmap learned FSMap version 28 2023-10-08T09:58:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.144+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f621d2bc00 con 0x55f621fd72c0 2023-10-08T09:58:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f621edfe50 con 0x55f621fd72c0 2023-10-08T09:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f859c00b0d0 con 0x55f621fd72c0 2023-10-08T09:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 20 client.16197 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621feb750 0x55f621fedb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 4 client.16197 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.145+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f621f10db0 con 0x55f621feb750 2023-10-08T09:58:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.148+0000 7f85a3fff700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621feb750 0x55f621fedb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.159+0000 7f85a3fff700 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621feb750 0x55f621fedb70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce03f70c1493f147 server_cookie=f1b58bf5c72cb in_seq=0 out_seq=0 2023-10-08T09:58:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.160+0000 7f85837fe700 10 client.16197 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.160+0000 7f85837fe700 1 -- 172.21.15.97:0/3106539762 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f621f10db0 con 0x55f621feb750 2023-10-08T09:58:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.160+0000 7f85837fe700 10 client.16197 handle_command_reply: tid=0 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.160+0000 7f85b689cb80 1 client.16197 shutdown 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.162+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 msgr2=0x55f621fe6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.162+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f859c000f80 tx=0x7f859c0099e0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 shutdown_connections 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd72c0 0x55f621fe6e20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd5e50 0x55f621fdf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621feb750 0x55f621fedb70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3106539762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd67b0 0x55f621fdfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85837fe700 0 client.16197 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.163+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 >> 172.21.15.97:0/3106539762 conn(0x55f621f39200 msgr2=0x55f621fbf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.164+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 shutdown_connections 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.164+0000 7f85b689cb80 1 -- 172.21.15.97:0/3106539762 wait complete. 2023-10-08T09:58:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.164+0000 7f85b689cb80 20 client.16197 trim_cache size 0 max 16384 2023-10-08T09:58:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.197+0000 7f85b689cb80 1 Processor -- start 2023-10-08T09:58:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.197+0000 7f85b689cb80 1 -- start start 2023-10-08T09:58:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd67b0 0x55f621fe6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd72c0 0x55f621fdedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621d988c0 con 0x55f621fd5e50 2023-10-08T09:58:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621fc0390 con 0x55f621fd67b0 2023-10-08T09:58:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85b689cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621fe08a0 con 0x55f621fd72c0 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58444/0 (socket says 172.21.15.97:58444) 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 -- 172.21.15.97:0/3196431888 learned_addr learned my addr 172.21.15.97:0/3196431888 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd72c0 0x55f621fdedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a3fff700 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd67b0 0x55f621fe6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 -- 172.21.15.97:0/3196431888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd72c0 msgr2=0x55f621fdedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd72c0 0x55f621fdedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 -- 172.21.15.97:0/3196431888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd67b0 msgr2=0x55f621fe6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd67b0 0x55f621fe6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.198+0000 7f85a37fe700 1 -- 172.21.15.97:0/3196431888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f621d691f0 con 0x55f621fd5e50 2023-10-08T09:58:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.199+0000 7f85a37fe700 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f859c007d00 tx=0x7f859c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9568b956c71f7cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.201+0000 7f858affd700 1 -- 172.21.15.97:0/3196431888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00e040 con 0x55f621fd5e50 2023-10-08T09:58:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.201+0000 7f858affd700 1 -- 172.21.15.97:0/3196431888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f859c026030 con 0x55f621fd5e50 2023-10-08T09:58:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.201+0000 7f858affd700 1 -- 172.21.15.97:0/3196431888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c01de80 con 0x55f621fd5e50 2023-10-08T09:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.203+0000 7f85b689cb80 1 -- 172.21.15.97:0/3196431888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 msgr2=0x55f621fedb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:58:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.204+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f859c007d00 tx=0x7f859c022040 comp rx=0 tx=0).stop 2023-10-08T09:58:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.204+0000 7f85b689cb80 1 -- 172.21.15.97:0/3196431888 shutdown_connections 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.204+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd72c0 0x55f621fdedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd5e50 0x55f621fedb40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 --2- 172.21.15.97:0/3196431888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd67b0 0x55f621fe6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 -- 172.21.15.97:0/3196431888 >> 172.21.15.97:0/3196431888 conn(0x55f621f39200 msgr2=0x55f621ff2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 -- 172.21.15.97:0/3196431888 shutdown_connections 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 -- 172.21.15.97:0/3196431888 wait complete. 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 -- start start 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.205+0000 7f85b689cb80 1 Processor -- start 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fe4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fe4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fe5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621fe6110 con 0x55f621fd67b0 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621fe6280 con 0x55f621fd72c0 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f621fe6440 con 0x55f621fd5e50 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85b689cb80 10 client.0 _ll_register_callbacks cb 0x55f621d117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fe5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fe5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38422/0 (socket says 172.21.15.97:38422) 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a37fe700 1 -- 172.21.15.97:0/2256286215 learned_addr learned my addr 172.21.15.97:0/2256286215 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fe4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a3fff700 1 -- 172.21.15.97:0/2256286215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 msgr2=0x55f621fe4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.206+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fe4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.207+0000 7f85a3fff700 1 -- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 msgr2=0x55f621fe5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.207+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fe5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:58:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.207+0000 7f85a3fff700 1 -- 172.21.15.97:0/2256286215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f621d2bc00 con 0x55f621fd67b0 2023-10-08T09:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.209+0000 7f85a3fff700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fe4cb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f85980073b0 tx=0x7f8598012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ff659cf6d67577 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.209+0000 7f85837fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.209+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598023050 con 0x55f621fd67b0 2023-10-08T09:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.209+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8598025430 con 0x55f621fd67b0 2023-10-08T09:58:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.210+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8598012c10 con 0x55f621fd67b0 2023-10-08T09:58:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.210+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85980125f0 con 0x55f621fd67b0 2023-10-08T09:58:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.210+0000 7f85b689cb80 10 client.16203 fetch_fsmap learned FSMap version 28 2023-10-08T09:58:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.210+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f621d41250 con 0x55f621fd67b0 2023-10-08T09:58:37.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.210+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8598012c10 con 0x55f621fd67b0 2023-10-08T09:58:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.212+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f859802b020 con 0x55f621fd67b0 2023-10-08T09:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 10 client.16203 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 20 client.16203 populate_metadata read hostname 'smithi097' 2023-10-08T09:58:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621fc2680 0x55f621fc4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:58:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 4 client.16203 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:58:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.219+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f621f10ad0 con 0x55f621fc2680 2023-10-08T09:58:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.220+0000 7f85a37fe700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621fc2680 0x55f621fc4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.222+0000 7f85837fe700 10 client.16203 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:58:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T09:58:37.222+0000 7f85a37fe700 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621fc2680 0x55f621fc4aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467dc8863df9d7a3 server_cookie=dfd8649a700b9a42 in_seq=0 out_seq=0 2023-10-08T09:58:37.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:37 smithi097 ceph-mon[99491]: pgmap v404: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 8.28k op/s 2023-10-08T09:58:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:37 smithi178 ceph-mon[103156]: pgmap v404: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 8.28k op/s 2023-10-08T09:58:37.779 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:37 smithi186 ceph-mon[104393]: pgmap v404: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 8.28k op/s 2023-10-08T09:58:38.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:38 smithi097 ceph-mon[99491]: pgmap v405: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 6.03k op/s 2023-10-08T09:58:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:38 smithi178 ceph-mon[103156]: pgmap v405: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 6.03k op/s 2023-10-08T09:58:38.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:38 smithi186 ceph-mon[104393]: pgmap v405: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 6.03k op/s 2023-10-08T09:58:41.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:41 smithi097 ceph-mon[99491]: pgmap v406: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 6.42k op/s 2023-10-08T09:58:41.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:41 smithi178 ceph-mon[103156]: pgmap v406: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 6.42k op/s 2023-10-08T09:58:41.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:41 smithi186 ceph-mon[104393]: pgmap v406: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 6.42k op/s 2023-10-08T09:58:43.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:43 smithi097 ceph-mon[99491]: pgmap v407: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 8.09k op/s 2023-10-08T09:58:43.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:43 smithi178 ceph-mon[103156]: pgmap v407: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 8.09k op/s 2023-10-08T09:58:43.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:43 smithi186 ceph-mon[104393]: pgmap v407: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 33 MiB/s wr, 8.09k op/s 2023-10-08T09:58:45.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:45 smithi178 ceph-mon[103156]: pgmap v408: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:58:45.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:45 smithi097 ceph-mon[99491]: pgmap v408: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:58:45.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:45 smithi186 ceph-mon[104393]: pgmap v408: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:58:47.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:47 smithi097 ceph-mon[99491]: pgmap v409: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 7.29k op/s 2023-10-08T09:58:47.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:47 smithi178 ceph-mon[103156]: pgmap v409: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 7.29k op/s 2023-10-08T09:58:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:47 smithi186 ceph-mon[104393]: pgmap v409: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 7.29k op/s 2023-10-08T09:58:48.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:48 smithi178 ceph-mon[103156]: pgmap v410: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 26 MiB/s wr, 6.50k op/s 2023-10-08T09:58:48.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:48 smithi186 ceph-mon[104393]: pgmap v410: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 26 MiB/s wr, 6.50k op/s 2023-10-08T09:58:48.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:48 smithi097 ceph-mon[99491]: pgmap v410: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 26 MiB/s wr, 6.50k op/s 2023-10-08T09:58:51.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:51 smithi097 ceph-mon[99491]: pgmap v411: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 6.51k op/s 2023-10-08T09:58:51.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:51 smithi178 ceph-mon[103156]: pgmap v411: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 6.51k op/s 2023-10-08T09:58:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:51 smithi186 ceph-mon[104393]: pgmap v411: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 26 MiB/s wr, 6.51k op/s 2023-10-08T09:58:53.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:53 smithi178 ceph-mon[103156]: pgmap v412: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 7.71k op/s 2023-10-08T09:58:53.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:53 smithi186 ceph-mon[104393]: pgmap v412: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 7.71k op/s 2023-10-08T09:58:53.842 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:53 smithi097 ceph-mon[99491]: pgmap v412: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 7.71k op/s 2023-10-08T09:58:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:55 smithi178 ceph-mon[103156]: pgmap v413: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2023-10-08T09:58:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:55 smithi186 ceph-mon[104393]: pgmap v413: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2023-10-08T09:58:56.095 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:55 smithi097 ceph-mon[99491]: pgmap v413: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 6.03k op/s 2023-10-08T09:58:57.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:57 smithi178 ceph-mon[103156]: pgmap v414: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 6.72k op/s 2023-10-08T09:58:57.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:57 smithi097 ceph-mon[99491]: pgmap v414: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 6.72k op/s 2023-10-08T09:58:57.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:57 smithi186 ceph-mon[104393]: pgmap v414: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 27 MiB/s wr, 6.72k op/s 2023-10-08T09:58:59.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:58:59 smithi097 ceph-mon[99491]: pgmap v415: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 7.08k op/s 2023-10-08T09:58:59.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:58:59 smithi178 ceph-mon[103156]: pgmap v415: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 7.08k op/s 2023-10-08T09:58:59.781 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:58:59 smithi186 ceph-mon[104393]: pgmap v415: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 7.08k op/s 2023-10-08T09:59:01.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:01 smithi097 ceph-mon[99491]: pgmap v416: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 5.87k op/s 2023-10-08T09:59:01.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:01 smithi178 ceph-mon[103156]: pgmap v416: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 5.87k op/s 2023-10-08T09:59:01.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:01 smithi186 ceph-mon[104393]: pgmap v416: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 5.87k op/s 2023-10-08T09:59:02.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:02 smithi186 ceph-mon[104393]: pgmap v417: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2023-10-08T09:59:02.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:02 smithi097 ceph-mon[99491]: pgmap v417: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2023-10-08T09:59:02.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:02 smithi178 ceph-mon[103156]: pgmap v417: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 28 MiB/s wr, 7.02k op/s 2023-10-08T09:59:04.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:59:05.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:05 smithi178 ceph-mon[103156]: pgmap v418: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 5.88k op/s 2023-10-08T09:59:05.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:05 smithi097 ceph-mon[99491]: pgmap v418: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 5.88k op/s 2023-10-08T09:59:05.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:05 smithi186 ceph-mon[104393]: pgmap v418: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 MiB/s wr, 5.88k op/s 2023-10-08T09:59:07.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:07 smithi097 ceph-mon[99491]: pgmap v419: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:59:07.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:07 smithi178 ceph-mon[103156]: pgmap v419: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:59:07.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:07 smithi186 ceph-mon[104393]: pgmap v419: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 27 MiB/s wr, 6.68k op/s 2023-10-08T09:59:09.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:09 smithi097 ceph-mon[99491]: pgmap v420: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 6.33k op/s 2023-10-08T09:59:09.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:09 smithi178 ceph-mon[103156]: pgmap v420: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 6.33k op/s 2023-10-08T09:59:09.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:09 smithi186 ceph-mon[104393]: pgmap v420: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 26 MiB/s wr, 6.33k op/s 2023-10-08T09:59:11.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:11 smithi097 ceph-mon[99491]: pgmap v421: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T09:59:11.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:11 smithi178 ceph-mon[103156]: pgmap v421: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T09:59:11.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:11 smithi186 ceph-mon[104393]: pgmap v421: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T09:59:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 -- 172.21.15.97:0/1814662228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 msgr2=0x7fb9d00ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:59:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 --2- 172.21.15.97:0/1814662228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d00ffaf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800b0d0 tx=0x7fb9c800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 -- 172.21.15.97:0/1814662228 shutdown_connections 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 --2- 172.21.15.97:0/1814662228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 0x7fb9d0107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 --2- 172.21.15.97:0/1814662228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d00ffaf0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 --2- 172.21.15.97:0/1814662228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 0x7fb9d0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 -- 172.21.15.97:0/1814662228 >> 172.21.15.97:0/1814662228 conn(0x7fb9d00fac40 msgr2=0x7fb9d00fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 -- 172.21.15.97:0/1814662228 shutdown_connections 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.132+0000 7fb9d8511700 1 -- 172.21.15.97:0/1814662228 wait complete. 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 Processor -- start 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 -- start start 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 0x7fb9d0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 0x7fb9d019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d019d320 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d019d460 con 0x7fb9d01008c0 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d8511700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d019d5f0 con 0x7fb9d0101260 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d62ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d62ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d01957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58462/0 (socket says 172.21.15.97:58462) 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d62ad700 1 -- 172.21.15.97:0/402293139 learned_addr learned my addr 172.21.15.97:0/402293139 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 0x7fb9d0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.133+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 0x7fb9d019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 -- 172.21.15.97:0/402293139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 msgr2=0x7fb9d019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 0x7fb9d019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 -- 172.21.15.97:0/402293139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 msgr2=0x7fb9d0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 0x7fb9d0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 -- 172.21.15.97:0/402293139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009d00 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01008c0 0x7fb9d0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9d0101260 0x7fb9d019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d00ff6d0 0x7fb9d01957f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8000f80 tx=0x7fb9c8008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9c77fe700 1 -- 172.21.15.97:0/402293139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800e040 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9c77fe700 1 -- 172.21.15.97:0/402293139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9c801e070 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9c77fe700 1 -- 172.21.15.97:0/402293139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80258e0 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d8511700 1 -- 172.21.15.97:0/402293139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d0068a10 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.134+0000 7fb9d8511700 1 -- 172.21.15.97:0/402293139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d0068ed0 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.136+0000 7fb9c77fe700 1 -- 172.21.15.97:0/402293139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb9c801ab10 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.137+0000 7fb9c77fe700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9bc0a1480 0x7fb9bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.137+0000 7fb9c77fe700 1 -- 172.21.15.97:0/402293139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb9c8020070 con 0x7fb9d00ff6d0 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.137+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9bc0a1480 0x7fb9bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.137+0000 7fb9e46fab80 1 Processor -- start 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/402293139 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fb9bc0a1480 0x7fb9bc0a3940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0011490 tx=0x7fb9c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 -- start start 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 0x5587e13e9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 0x5587e13ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e1351460 con 0x5587e13ea800 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e1342c20 con 0x5587e13e9060 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e1340540 con 0x5587e13e99c0 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d5aac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 0x5587e13ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58474/0 (socket says 172.21.15.97:58474) 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d5aac700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 0x5587e13ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60172/0 (socket says 172.21.15.97:60172) 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1387808736 learned_addr learned my addr 172.21.15.97:0/1387808736 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 0x5587e13e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1387808736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 msgr2=0x5587e13ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 0x5587e13ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1387808736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 msgr2=0x5587e13e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:59:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 0x5587e13e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1387808736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587e1166de0 con 0x5587e13ea800 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 0x5587e13e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.138+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011540 tx=0x7fb9cc011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a17981ebdab4962 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9c4ff9700 1 -- 172.21.15.97:0/1387808736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc019630 con 0x5587e13ea800 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9c4ff9700 1 -- 172.21.15.97:0/1387808736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9cc019c50 con 0x5587e13ea800 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9c4ff9700 1 -- 172.21.15.97:0/1387808736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00ec10 con 0x5587e13ea800 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1387808736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 msgr2=0x5587e13eac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011540 tx=0x7fb9cc011850 comp rx=0 tx=0).stop 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1387808736 shutdown_connections 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13eac90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e9060 0x5587e13e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 --2- 172.21.15.97:0/1387808736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e99c0 0x5587e13ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.139+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1387808736 >> 172.21.15.97:0/1387808736 conn(0x5587e135d260 msgr2=0x5587e13e3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1387808736 shutdown_connections 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1387808736 wait complete. 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- start start 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 Processor -- start 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e9060 0x5587e13f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e99c0 0x5587e13f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e133be30 con 0x5587e13ea800 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e12d8710 con 0x5587e13e99c0 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587e12d83b0 con 0x5587e13e9060 2023-10-08T09:59:12.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9e46fab80 10 client.0 _ll_register_callbacks cb 0x5587e1135800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9d6aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9d62ad700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e9060 0x5587e13f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9d6aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13fa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58480/0 (socket says 172.21.15.97:58480) 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1715202213 learned_addr learned my addr 172.21.15.97:0/1715202213 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.140+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e99c0 0x5587e13f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1715202213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e9060 msgr2=0x5587e13f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e9060 0x5587e13f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e99c0 msgr2=0x5587e13f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e99c0 0x5587e13f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 -- 172.21.15.97:0/1715202213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc009dd0 con 0x5587e13ea800 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d5aac700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5587e13e99c0 0x5587e13f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5587e13e9060 0x5587e13f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:59:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9d6aae700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5587e13ea800 0x5587e13fa3b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc004690 tx=0x7fb9cc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfbcb7cc3eb929b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 1 -- 172.21.15.97:0/1715202213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc019840 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 1 -- 172.21.15.97:0/1715202213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb9cc0199a0 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1715202213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587e1303eb0 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 1 -- 172.21.15.97:0/1715202213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9cc00eea0 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 10 client.16215 fetch_fsmap learned FSMap version 28 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1715202213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587e114fc60 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 1 -- 172.21.15.97:0/1715202213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587e1303eb0 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9b9ffb700 1 -- 172.21.15.97:0/1715202213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fb9cc029020 con 0x5587e13ea800 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:59:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 20 client.16215 populate_metadata read hostname 'smithi097' 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5587e13fec00 0x5587e1401020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.141+0000 7fb9e46fab80 4 client.16215 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.142+0000 7fb9e46fab80 1 -- 172.21.15.97:0/1715202213 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587e1334e10 con 0x5587e13fec00 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.142+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5587e13fec00 0x5587e1401020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.142+0000 7fb9b9ffb700 10 client.16215 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T09:59:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T09:59:12.142+0000 7fb9d62ad700 1 --2- 172.21.15.97:0/1715202213 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5587e13fec00 0x5587e1401020 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6394dee5f719cee0 server_cookie=86f6752e2136bf47 in_seq=0 out_seq=0 2023-10-08T09:59:13.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:13 smithi178 ceph-mon[103156]: pgmap v422: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 5.59k op/s 2023-10-08T09:59:13.714 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:13 smithi097 ceph-mon[99491]: pgmap v422: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 5.59k op/s 2023-10-08T09:59:13.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:13 smithi186 ceph-mon[104393]: pgmap v422: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 5.59k op/s 2023-10-08T09:59:15.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:15 smithi097 ceph-mon[99491]: pgmap v423: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T09:59:15.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:15 smithi178 ceph-mon[103156]: pgmap v423: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T09:59:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:15 smithi186 ceph-mon[104393]: pgmap v423: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T09:59:16.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:16 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:59:16.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:16 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:59:16.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:16 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:59:17.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:17 smithi097 ceph-mon[99491]: pgmap v424: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2023-10-08T09:59:17.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:17 smithi178 ceph-mon[103156]: pgmap v424: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2023-10-08T09:59:17.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:17 smithi186 ceph-mon[104393]: pgmap v424: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 3.94k op/s 2023-10-08T09:59:19.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:19 smithi097 ceph-mon[99491]: pgmap v425: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2023-10-08T09:59:19.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:19 smithi178 ceph-mon[103156]: pgmap v425: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2023-10-08T09:59:19.894 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:19 smithi186 ceph-mon[104393]: pgmap v425: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 3.77k op/s 2023-10-08T09:59:21.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:21 smithi097 ceph-mon[99491]: pgmap v426: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2023-10-08T09:59:21.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:21 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:59:21.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:21 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:59:21.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:21 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:59:21.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:21 smithi178 ceph-mon[103156]: pgmap v426: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2023-10-08T09:59:21.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:21 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:59:21.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:21 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:59:21.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:21 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:59:21.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:21 smithi186 ceph-mon[104393]: pgmap v426: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2023-10-08T09:59:21.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:21 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:59:21.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:21 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:59:21.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:21 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T09:59:23.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:23 smithi178 ceph-mon[103156]: pgmap v427: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 4.12k op/s 2023-10-08T09:59:23.710 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:23 smithi097 ceph-mon[99491]: pgmap v427: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 4.12k op/s 2023-10-08T09:59:23.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:23 smithi186 ceph-mon[104393]: pgmap v427: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 4.12k op/s 2023-10-08T09:59:25.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:25 smithi097 ceph-mon[99491]: pgmap v428: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T09:59:25.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:25 smithi178 ceph-mon[103156]: pgmap v428: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T09:59:25.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:25 smithi186 ceph-mon[104393]: pgmap v428: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T09:59:27.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:27 smithi097 ceph-mon[99491]: pgmap v429: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2023-10-08T09:59:27.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:27 smithi178 ceph-mon[103156]: pgmap v429: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2023-10-08T09:59:27.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:27 smithi186 ceph-mon[104393]: pgmap v429: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.92k op/s 2023-10-08T09:59:29.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:29 smithi097 ceph-mon[99491]: pgmap v430: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2023-10-08T09:59:29.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:29 smithi178 ceph-mon[103156]: pgmap v430: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2023-10-08T09:59:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:29 smithi186 ceph-mon[104393]: pgmap v430: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 4.56k op/s 2023-10-08T09:59:31.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:31 smithi097 ceph-mon[99491]: pgmap v431: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-10-08T09:59:31.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:31 smithi178 ceph-mon[103156]: pgmap v431: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-10-08T09:59:31.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:31 smithi186 ceph-mon[104393]: pgmap v431: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-10-08T09:59:33.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:33 smithi097 ceph-mon[99491]: pgmap v432: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 5.15k op/s 2023-10-08T09:59:33.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:33 smithi178 ceph-mon[103156]: pgmap v432: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 5.15k op/s 2023-10-08T09:59:33.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:33 smithi186 ceph-mon[104393]: pgmap v432: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 5.15k op/s 2023-10-08T09:59:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:35 smithi186 ceph-mon[104393]: pgmap v433: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T09:59:35.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:35 smithi097 ceph-mon[99491]: pgmap v433: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T09:59:35.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:35 smithi178 ceph-mon[103156]: pgmap v433: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T09:59:36.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:36 smithi097 ceph-mon[99491]: pgmap v434: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2023-10-08T09:59:36.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:36 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:36 smithi178 ceph-mon[103156]: pgmap v434: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2023-10-08T09:59:36.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:36 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:36 smithi186 ceph-mon[104393]: pgmap v434: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2023-10-08T09:59:37.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:36 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:39.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:39 smithi097 ceph-mon[99491]: pgmap v435: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 4.60k op/s 2023-10-08T09:59:39.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:39 smithi178 ceph-mon[103156]: pgmap v435: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 4.60k op/s 2023-10-08T09:59:39.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:39 smithi186 ceph-mon[104393]: pgmap v435: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 4.60k op/s 2023-10-08T09:59:41.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:41 smithi097 ceph-mon[99491]: pgmap v436: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.61k op/s 2023-10-08T09:59:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:41 smithi178 ceph-mon[103156]: pgmap v436: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.61k op/s 2023-10-08T09:59:41.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:41 smithi186 ceph-mon[104393]: pgmap v436: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 3.61k op/s 2023-10-08T09:59:43.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:43 smithi097 ceph-mon[99491]: pgmap v437: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T09:59:43.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:43 smithi178 ceph-mon[103156]: pgmap v437: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T09:59:43.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:43 smithi186 ceph-mon[104393]: pgmap v437: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T09:59:45.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:45 smithi097 ceph-mon[99491]: pgmap v438: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T09:59:45.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:45 smithi178 ceph-mon[103156]: pgmap v438: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T09:59:45.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:45 smithi186 ceph-mon[104393]: pgmap v438: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T09:59:47.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:47 smithi097 ceph-mon[99491]: pgmap v439: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:47.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:47 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:47 smithi178 ceph-mon[103156]: pgmap v439: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:47 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:47 smithi186 ceph-mon[104393]: pgmap v439: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:47.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:47 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:59:49.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:49 smithi178 ceph-mon[103156]: pgmap v440: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2023-10-08T09:59:49.710 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:49 smithi097 ceph-mon[99491]: pgmap v440: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2023-10-08T09:59:49.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:49 smithi186 ceph-mon[104393]: pgmap v440: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.77k op/s 2023-10-08T09:59:51.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:51 smithi097 ceph-mon[99491]: pgmap v441: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2023-10-08T09:59:51.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:51 smithi178 ceph-mon[103156]: pgmap v441: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2023-10-08T09:59:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:51 smithi186 ceph-mon[104393]: pgmap v441: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 4.07k op/s 2023-10-08T09:59:53.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:53 smithi097 ceph-mon[99491]: pgmap v442: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2023-10-08T09:59:53.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:53 smithi178 ceph-mon[103156]: pgmap v442: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2023-10-08T09:59:53.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:53 smithi186 ceph-mon[104393]: pgmap v442: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 5.23k op/s 2023-10-08T09:59:55.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:55 smithi097 ceph-mon[99491]: pgmap v443: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T09:59:55.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:55 smithi178 ceph-mon[103156]: pgmap v443: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T09:59:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:55 smithi186 ceph-mon[104393]: pgmap v443: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T09:59:57.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:57 smithi097 ceph-mon[99491]: pgmap v444: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:57.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:57 smithi178 ceph-mon[103156]: pgmap v444: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:57.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:57 smithi186 ceph-mon[104393]: pgmap v444: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-10-08T09:59:59.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 09:59:59 smithi178 ceph-mon[103156]: pgmap v445: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2023-10-08T09:59:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 09:59:59 smithi186 ceph-mon[104393]: pgmap v445: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2023-10-08T09:59:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 09:59:59 smithi097 ceph-mon[99491]: pgmap v445: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 4.82k op/s 2023-10-08T10:00:00.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:00 smithi186 ceph-mon[104393]: overall HEALTH_OK 2023-10-08T10:00:00.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:00 smithi097 ceph-mon[99491]: overall HEALTH_OK 2023-10-08T10:00:00.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:00 smithi178 ceph-mon[103156]: overall HEALTH_OK 2023-10-08T10:00:01.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:01 smithi186 ceph-mon[104393]: pgmap v446: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.10k op/s 2023-10-08T10:00:01.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:01 smithi097 ceph-mon[99491]: pgmap v446: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.10k op/s 2023-10-08T10:00:01.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:01 smithi178 ceph-mon[103156]: pgmap v446: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.10k op/s 2023-10-08T10:00:03.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:03 smithi186 ceph-mon[104393]: pgmap v447: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 5.36k op/s 2023-10-08T10:00:03.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:03 smithi097 ceph-mon[99491]: pgmap v447: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 5.36k op/s 2023-10-08T10:00:03.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:03 smithi178 ceph-mon[103156]: pgmap v447: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 5.36k op/s 2023-10-08T10:00:04.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:04 smithi097 ceph-mon[99491]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 4.20k op/s 2023-10-08T10:00:04.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:04 smithi178 ceph-mon[103156]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 4.20k op/s 2023-10-08T10:00:05.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:04 smithi186 ceph-mon[104393]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 4.20k op/s 2023-10-08T10:00:07.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:07 smithi097 ceph-mon[99491]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:00:07.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:07 smithi178 ceph-mon[103156]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:00:07.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:07 smithi186 ceph-mon[104393]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:00:09.594 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:09 smithi178 ceph-mon[103156]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:09.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:09 smithi097 ceph-mon[99491]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:09.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:09 smithi186 ceph-mon[104393]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:11.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:11 smithi097 ceph-mon[99491]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-10-08T10:00:11.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:11 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:00:11.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:11 smithi178 ceph-mon[103156]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-10-08T10:00:11.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:11 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:00:11.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:11 smithi186 ceph-mon[104393]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-10-08T10:00:11.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:11 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:00:13.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:13 smithi097 ceph-mon[99491]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.39k op/s 2023-10-08T10:00:13.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:13 smithi178 ceph-mon[103156]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.39k op/s 2023-10-08T10:00:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:13 smithi186 ceph-mon[104393]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.39k op/s 2023-10-08T10:00:15.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:15 smithi097 ceph-mon[99491]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:00:15.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:15 smithi178 ceph-mon[103156]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:00:15.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:15 smithi186 ceph-mon[104393]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:00:17.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:17 smithi097 ceph-mon[99491]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-10-08T10:00:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:17 smithi178 ceph-mon[103156]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-10-08T10:00:17.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:17 smithi186 ceph-mon[104393]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-10-08T10:00:19.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:19 smithi178 ceph-mon[103156]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:19.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:19 smithi097 ceph-mon[99491]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:19.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:19 smithi186 ceph-mon[104393]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 4.88k op/s 2023-10-08T10:00:21.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:21 smithi097 ceph-mon[99491]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:00:21.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:21 smithi178 ceph-mon[103156]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:00:21.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:21 smithi186 ceph-mon[104393]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:00:23.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:23 smithi097 ceph-mon[99491]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T10:00:23.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:23 smithi178 ceph-mon[103156]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T10:00:23.779 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:23 smithi186 ceph-mon[104393]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 5.14k op/s 2023-10-08T10:00:24.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:24 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:00:24.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:24 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:00:24.960 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:24 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:00:25.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:25 smithi186 ceph-mon[104393]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2023-10-08T10:00:25.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:25 smithi097 ceph-mon[99491]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2023-10-08T10:00:25.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:25 smithi178 ceph-mon[103156]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.55k op/s 2023-10-08T10:00:27.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:27 smithi186 ceph-mon[104393]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T10:00:27.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:27 smithi097 ceph-mon[99491]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T10:00:27.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:27 smithi178 ceph-mon[103156]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2023-10-08T10:00:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:29 smithi186 ceph-mon[104393]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2023-10-08T10:00:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:00:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:00:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:29 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:00:29.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:29 smithi097 ceph-mon[99491]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2023-10-08T10:00:29.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:00:29.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:00:29.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:29 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:00:29.959 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:29 smithi178 ceph-mon[103156]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 4.74k op/s 2023-10-08T10:00:29.959 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:00:29.959 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:00:29.959 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:29 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:00:31.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:31 smithi097 ceph-mon[99491]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T10:00:31.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:31 smithi178 ceph-mon[103156]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T10:00:31.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:31 smithi186 ceph-mon[104393]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.89k op/s 2023-10-08T10:00:33.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:33 smithi097 ceph-mon[99491]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.26k op/s 2023-10-08T10:00:33.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:33 smithi178 ceph-mon[103156]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.26k op/s 2023-10-08T10:00:33.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:33 smithi186 ceph-mon[104393]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.26k op/s 2023-10-08T10:00:35.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:35 smithi097 ceph-mon[99491]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2023-10-08T10:00:35.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:35 smithi178 ceph-mon[103156]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2023-10-08T10:00:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:35 smithi186 ceph-mon[104393]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.24k op/s 2023-10-08T10:00:37.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:37 smithi097 ceph-mon[99491]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:00:37.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:37 smithi178 ceph-mon[103156]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:00:37.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:37 smithi186 ceph-mon[104393]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:00:39.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:39 smithi097 ceph-mon[99491]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.20k op/s 2023-10-08T10:00:39.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:39 smithi178 ceph-mon[103156]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.20k op/s 2023-10-08T10:00:39.782 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:39 smithi186 ceph-mon[104393]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 5.20k op/s 2023-10-08T10:00:41.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:41 smithi097 ceph-mon[99491]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2023-10-08T10:00:41.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:41 smithi178 ceph-mon[103156]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2023-10-08T10:00:41.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:41 smithi186 ceph-mon[104393]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2023-10-08T10:00:43.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:43 smithi097 ceph-mon[99491]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2023-10-08T10:00:43.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:43 smithi178 ceph-mon[103156]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2023-10-08T10:00:43.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:43 smithi186 ceph-mon[104393]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.71k op/s 2023-10-08T10:00:45.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:45 smithi097 ceph-mon[99491]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-10-08T10:00:45.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:45 smithi178 ceph-mon[103156]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-10-08T10:00:45.781 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:45 smithi186 ceph-mon[104393]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-10-08T10:00:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:47 smithi186 ceph-mon[104393]: pgmap v469: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2023-10-08T10:00:47.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:47 smithi097 ceph-mon[99491]: pgmap v469: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2023-10-08T10:00:47.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:47 smithi178 ceph-mon[103156]: pgmap v469: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.58k op/s 2023-10-08T10:00:49.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:49 smithi186 ceph-mon[104393]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2023-10-08T10:00:49.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:49 smithi097 ceph-mon[99491]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2023-10-08T10:00:49.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:49 smithi178 ceph-mon[103156]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.70k op/s 2023-10-08T10:00:51.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:51 smithi097 ceph-mon[99491]: pgmap v471: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.93k op/s 2023-10-08T10:00:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:51 smithi178 ceph-mon[103156]: pgmap v471: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.93k op/s 2023-10-08T10:00:52.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:51 smithi186 ceph-mon[104393]: pgmap v471: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.93k op/s 2023-10-08T10:00:53.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:53 smithi097 ceph-mon[99491]: pgmap v472: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2023-10-08T10:00:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:53 smithi178 ceph-mon[103156]: pgmap v472: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2023-10-08T10:00:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:53 smithi186 ceph-mon[104393]: pgmap v472: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.01k op/s 2023-10-08T10:00:54.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:54 smithi097 ceph-mon[99491]: pgmap v473: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2023-10-08T10:00:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:54 smithi178 ceph-mon[103156]: pgmap v473: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2023-10-08T10:00:55.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:54 smithi186 ceph-mon[104393]: pgmap v473: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2023-10-08T10:00:57.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:57 smithi097 ceph-mon[99491]: pgmap v474: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:00:57.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:57 smithi178 ceph-mon[103156]: pgmap v474: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:00:57.779 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:57 smithi186 ceph-mon[104393]: pgmap v474: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:00:59.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:00:59 smithi097 ceph-mon[99491]: pgmap v475: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2023-10-08T10:00:59.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:00:59 smithi178 ceph-mon[103156]: pgmap v475: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2023-10-08T10:00:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:00:59 smithi186 ceph-mon[104393]: pgmap v475: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.80k op/s 2023-10-08T10:01:01.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:01 smithi097 ceph-mon[99491]: pgmap v476: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2023-10-08T10:01:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:01 smithi178 ceph-mon[103156]: pgmap v476: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2023-10-08T10:01:01.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:01 smithi186 ceph-mon[104393]: pgmap v476: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 4.09k op/s 2023-10-08T10:01:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:03 smithi097 ceph-mon[99491]: pgmap v477: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T10:01:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:03 smithi178 ceph-mon[103156]: pgmap v477: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T10:01:03.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:03 smithi186 ceph-mon[104393]: pgmap v477: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 5.02k op/s 2023-10-08T10:01:05.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:05 smithi097 ceph-mon[99491]: pgmap v478: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T10:01:05.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:05 smithi178 ceph-mon[103156]: pgmap v478: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T10:01:05.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:05 smithi186 ceph-mon[104393]: pgmap v478: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 4.08k op/s 2023-10-08T10:01:07.519 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-08T10:01:07.520 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/cephfs/filesystem.py", line 1731, in wait_until_scrub_complete out_json = self.rank_tell(["scrub", "status"], rank=rank) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/cephfs/filesystem.py", line 1300, in rank_tell out = self.get_ceph_cmd_stdout("tell", f"mds.{self.id}:{rank}", *command) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/ceph_manager.py", line 1592, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status' 2023-10-08T10:01:07.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:07 smithi097 ceph-mon[99491]: pgmap v479: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-10-08T10:01:07.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:07 smithi178 ceph-mon[103156]: pgmap v479: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-10-08T10:01:07.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:07 smithi186 ceph-mon[104393]: pgmap v479: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-10-08T10:01:09.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:09 smithi186 ceph-mon[104393]: pgmap v480: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:09.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:09 smithi097 ceph-mon[99491]: pgmap v480: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:09.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:09 smithi178 ceph-mon[103156]: pgmap v480: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:11.774 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:11 smithi186 ceph-mon[104393]: pgmap v481: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-10-08T10:01:11.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:11 smithi097 ceph-mon[99491]: pgmap v481: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-10-08T10:01:11.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:11 smithi178 ceph-mon[103156]: pgmap v481: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-10-08T10:01:13.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:13 smithi186 ceph-mon[104393]: pgmap v482: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2023-10-08T10:01:13.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:13 smithi097 ceph-mon[99491]: pgmap v482: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2023-10-08T10:01:13.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:13 smithi178 ceph-mon[103156]: pgmap v482: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.29k op/s 2023-10-08T10:01:15.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:15 smithi097 ceph-mon[99491]: pgmap v483: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2023-10-08T10:01:15.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:15 smithi178 ceph-mon[103156]: pgmap v483: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2023-10-08T10:01:16.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:15 smithi186 ceph-mon[104393]: pgmap v483: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2023-10-08T10:01:17.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:17 smithi097 ceph-mon[99491]: pgmap v484: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T10:01:17.957 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:17 smithi178 ceph-mon[103156]: pgmap v484: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T10:01:18.033 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:17 smithi186 ceph-mon[104393]: pgmap v484: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.86k op/s 2023-10-08T10:01:19.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:19 smithi097 ceph-mon[99491]: pgmap v485: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:19.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:19 smithi178 ceph-mon[103156]: pgmap v485: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:20.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:19 smithi186 ceph-mon[104393]: pgmap v485: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:21.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:21 smithi097 ceph-mon[99491]: pgmap v486: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T10:01:21.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:21 smithi178 ceph-mon[103156]: pgmap v486: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T10:01:22.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:21 smithi186 ceph-mon[104393]: pgmap v486: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.97k op/s 2023-10-08T10:01:23.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:23 smithi097 ceph-mon[99491]: pgmap v487: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.09k op/s 2023-10-08T10:01:23.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:23 smithi178 ceph-mon[103156]: pgmap v487: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.09k op/s 2023-10-08T10:01:24.030 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:23 smithi186 ceph-mon[104393]: pgmap v487: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 4.09k op/s 2023-10-08T10:01:24.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:24 smithi097 ceph-mon[99491]: pgmap v488: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:01:24.958 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:24 smithi178 ceph-mon[103156]: pgmap v488: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:01:25.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:24 smithi186 ceph-mon[104393]: pgmap v488: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.14k op/s 2023-10-08T10:01:25.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:25 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:25.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:25 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:26.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:25 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:26.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:26 smithi097 ceph-mon[99491]: pgmap v489: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.34k op/s 2023-10-08T10:01:26.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:26 smithi178 ceph-mon[103156]: pgmap v489: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.34k op/s 2023-10-08T10:01:27.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:26 smithi186 ceph-mon[104393]: pgmap v489: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.34k op/s 2023-10-08T10:01:29.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:29 smithi186 ceph-mon[104393]: pgmap v490: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:01:29.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:29 smithi097 ceph-mon[99491]: pgmap v490: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:01:29.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:29 smithi178 ceph-mon[103156]: pgmap v490: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 4.25k op/s 2023-10-08T10:01:30.528 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:30 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:01:30.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:30 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:01:30.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:30 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:01:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:31 smithi186 ceph-mon[104393]: pgmap v491: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2023-10-08T10:01:31.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:31 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:31 smithi097 ceph-mon[99491]: pgmap v491: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2023-10-08T10:01:31.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:31 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:31.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:31 smithi178 ceph-mon[103156]: pgmap v491: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2023-10-08T10:01:31.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:31 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:33.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:33 smithi097 ceph-mon[99491]: pgmap v492: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2023-10-08T10:01:33.709 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:33 smithi178 ceph-mon[103156]: pgmap v492: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2023-10-08T10:01:33.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:33 smithi186 ceph-mon[104393]: pgmap v492: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 3.69k op/s 2023-10-08T10:01:34.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO) 2023-10-08T10:01:34.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:34.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO) 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:01:34.706 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:01:34.707 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:01:34.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO) 2023-10-08T10:01:34.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:34.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:34.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:01:34.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:01:34.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:01:35.955 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:35 smithi097 ceph-mon[99491]: pgmap v493: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-10-08T10:01:35.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:35 smithi178 ceph-mon[103156]: pgmap v493: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-10-08T10:01:36.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:35 smithi186 ceph-mon[104393]: pgmap v493: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-10-08T10:01:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:01:37.217+0000 7f858bfff700 1 -- 172.21.15.97:0/3198039455 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f857c003550 con 0x7f85a41a66e0 2023-10-08T10:01:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:01:37.217+0000 7f85a17fa700 1 -- 172.21.15.97:0/3198039455 <== mon.2 v2:172.21.15.178:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (secure 0 0 0) 0x7f857c003550 con 0x7f85a41a66e0 2023-10-08T10:01:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:01:37.331+0000 7f85827fc700 1 -- 172.21.15.97:0/2256286215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8574003550 con 0x55f621fd67b0 2023-10-08T10:01:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:01:37.332+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mon.0 v2:172.21.15.97:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 409+0+0 (secure 0 0 0) 0x7f8574003550 con 0x55f621fd67b0 2023-10-08T10:01:37.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:37 smithi097 ceph-mon[99491]: pgmap v494: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2023-10-08T10:01:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:37 smithi178 ceph-mon[103156]: pgmap v494: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2023-10-08T10:01:38.027 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:37 smithi186 ceph-mon[104393]: pgmap v494: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2023-10-08T10:01:38.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:38 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:38.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:38 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:38.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:38 smithi097 ceph-mon[99491]: pgmap v495: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-10-08T10:01:38.963 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:38 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:38.963 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:38 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:38.963 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:38 smithi178 ceph-mon[103156]: pgmap v495: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-10-08T10:01:39.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:38 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:39.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:38 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:39.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:38 smithi186 ceph-mon[104393]: pgmap v495: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-10-08T10:01:39.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:39 smithi097 ceph-mon[99491]: Health check failed: 0 slow ops, oldest one blocked for 184 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:39.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:39 smithi178 ceph-mon[103156]: Health check failed: 0 slow ops, oldest one blocked for 184 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:39 smithi186 ceph-mon[104393]: Health check failed: 0 slow ops, oldest one blocked for 184 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:40.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:40 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:40.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:40 smithi097 ceph-mon[99491]: pgmap v496: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:01:40.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:40 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:40.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:40 smithi178 ceph-mon[103156]: pgmap v496: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:01:41.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:40 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:41.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:40 smithi186 ceph-mon[104393]: pgmap v496: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-10-08T10:01:43.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:42 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:43.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:42 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:43.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:42 smithi186 ceph-mon[104393]: pgmap v497: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2023-10-08T10:01:43.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:42 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:43.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:42 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:43.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:42 smithi097 ceph-mon[99491]: pgmap v497: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2023-10-08T10:01:43.210 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:42 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:43.210 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:42 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:43.210 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:42 smithi178 ceph-mon[103156]: pgmap v497: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2023-10-08T10:01:44.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:44 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 190 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:44.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:44 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 190 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:44.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:44 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 190 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:45.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:45 smithi097 ceph-mon[99491]: pgmap v498: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2023-10-08T10:01:45.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:45 smithi178 ceph-mon[103156]: pgmap v498: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2023-10-08T10:01:45.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:45 smithi186 ceph-mon[104393]: pgmap v498: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 4.96k op/s 2023-10-08T10:01:47.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:47 smithi097 ceph-mon[99491]: pgmap v499: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:01:47.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:47 smithi178 ceph-mon[103156]: pgmap v499: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:01:47.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:47 smithi186 ceph-mon[104393]: pgmap v499: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:01:49.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:49 smithi097 ceph-mon[99491]: pgmap v500: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 MiB/s wr, 4.95k op/s 2023-10-08T10:01:49.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:49 smithi178 ceph-mon[103156]: pgmap v500: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 MiB/s wr, 4.95k op/s 2023-10-08T10:01:49.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:49 smithi186 ceph-mon[104393]: pgmap v500: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 MiB/s wr, 4.95k op/s 2023-10-08T10:01:50.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:50 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:50.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:50 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:50.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:50 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 195 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:50.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:50 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:50.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:50 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:50.707 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:50 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 195 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:50 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:50 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:50.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:50 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 195 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:51.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:51 smithi186 ceph-mon[104393]: pgmap v501: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-10-08T10:01:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:51 smithi097 ceph-mon[99491]: pgmap v501: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-10-08T10:01:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:51 smithi178 ceph-mon[103156]: pgmap v501: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-10-08T10:01:53.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:53 smithi186 ceph-mon[104393]: pgmap v502: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:01:53.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:53 smithi097 ceph-mon[99491]: pgmap v502: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:01:53.959 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:53 smithi178 ceph-mon[103156]: pgmap v502: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.37k op/s 2023-10-08T10:01:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:54 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:54 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:54.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:54 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:54.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:54 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:54 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:54 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:55.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:55 smithi186 ceph-mon[104393]: pgmap v503: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.46k op/s 2023-10-08T10:01:55.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:55 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 201 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:55.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:55 smithi097 ceph-mon[99491]: pgmap v503: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.46k op/s 2023-10-08T10:01:55.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:55 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 201 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:55.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:55 smithi178 ceph-mon[103156]: pgmap v503: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 5.46k op/s 2023-10-08T10:01:55.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:55 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 201 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:01:57.778 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:57 smithi186 ceph-mon[104393]: pgmap v504: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.84k op/s 2023-10-08T10:01:57.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:57 smithi097 ceph-mon[99491]: pgmap v504: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.84k op/s 2023-10-08T10:01:57.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:57 smithi178 ceph-mon[103156]: pgmap v504: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 3.84k op/s 2023-10-08T10:01:58.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:58 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:58.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:58 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:58 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:58 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:58 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:01:59.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:58 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:01:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:01:59 smithi097 ceph-mon[99491]: pgmap v505: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:01:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:01:59 smithi178 ceph-mon[103156]: pgmap v505: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:02:00.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:01:59 smithi186 ceph-mon[104393]: pgmap v505: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 4.67k op/s 2023-10-08T10:02:01.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:01 smithi097 ceph-mon[99491]: pgmap v506: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:02:01.705 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:01 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 207 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:01 smithi178 ceph-mon[103156]: pgmap v506: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:02:01.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:01 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 207 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:01.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:01 smithi186 ceph-mon[104393]: pgmap v506: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2023-10-08T10:02:01.780 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:01 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 207 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:02.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:02 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:02.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:02 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:02.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:02 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:02.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:02 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:02 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:03.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:02 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:03.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:03 smithi097 ceph-mon[99491]: pgmap v507: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.35k op/s 2023-10-08T10:02:03.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:03 smithi178 ceph-mon[103156]: pgmap v507: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.35k op/s 2023-10-08T10:02:04.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:03 smithi186 ceph-mon[104393]: pgmap v507: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 3.35k op/s 2023-10-08T10:02:05.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:05 smithi178 ceph-mon[103156]: pgmap v508: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.60k op/s 2023-10-08T10:02:05.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:05 smithi097 ceph-mon[99491]: pgmap v508: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.60k op/s 2023-10-08T10:02:06.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:05 smithi186 ceph-mon[104393]: pgmap v508: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.60k op/s 2023-10-08T10:02:06.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:06 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:06.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:06 smithi097 ceph-mon[99491]: pgmap v509: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2023-10-08T10:02:06.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:06 smithi097 ceph-mon[99491]: scrub summary: active paths [/] 2023-10-08T10:02:06.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:06 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:06.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:06 smithi178 ceph-mon[103156]: pgmap v509: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2023-10-08T10:02:06.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:06 smithi178 ceph-mon[103156]: scrub summary: active paths [/] 2023-10-08T10:02:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:06 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:06 smithi186 ceph-mon[104393]: pgmap v509: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.41k op/s 2023-10-08T10:02:07.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:06 smithi186 ceph-mon[104393]: scrub summary: active paths [/] 2023-10-08T10:02:07.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:07 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 212 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:07.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:07 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 212 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:08.029 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:07 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 212 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:08.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:08 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:08.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:08 smithi097 ceph-mon[99491]: pgmap v510: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.35k op/s 2023-10-08T10:02:08.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:08 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:08.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:08 smithi178 ceph-mon[103156]: pgmap v510: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.35k op/s 2023-10-08T10:02:09.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:08 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:09.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:08 smithi186 ceph-mon[104393]: pgmap v510: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.35k op/s 2023-10-08T10:02:11.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:10 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:11.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:10 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:11.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:10 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:11.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:10 smithi186 ceph-mon[104393]: pgmap v511: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-10-08T10:02:11.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:10 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:11.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:10 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:11.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:10 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:11.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:10 smithi097 ceph-mon[99491]: pgmap v511: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-10-08T10:02:11.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:10 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:11.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:10 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:11.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:10 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:11.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:10 smithi178 ceph-mon[103156]: pgmap v511: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-10-08T10:02:12.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:11 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:12.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:11 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:12.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:11 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:13.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:12 smithi097 ceph-mon[99491]: pgmap v512: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 3.95k op/s 2023-10-08T10:02:13.208 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:12 smithi178 ceph-mon[103156]: pgmap v512: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 3.95k op/s 2023-10-08T10:02:13.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:12 smithi186 ceph-mon[104393]: pgmap v512: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 3.95k op/s 2023-10-08T10:02:14.204 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:13 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 218 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:14.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:13 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 218 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:14.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:13 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 218 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:15.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:14 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:15.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:14 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:15.205 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:14 smithi178 ceph-mon[103156]: pgmap v513: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-10-08T10:02:15.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:14 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:15.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:14 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:15.208 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:14 smithi097 ceph-mon[99491]: pgmap v513: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-10-08T10:02:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:14 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:14 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:15.278 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:14 smithi186 ceph-mon[104393]: pgmap v513: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-10-08T10:02:17.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:17 smithi097 ceph-mon[99491]: pgmap v514: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2023-10-08T10:02:17.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:17 smithi178 ceph-mon[103156]: pgmap v514: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2023-10-08T10:02:17.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:17 smithi186 ceph-mon[104393]: pgmap v514: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 3.91k op/s 2023-10-08T10:02:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:18 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:18.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:18 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:18.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:18 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:18.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:18 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:18.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:18 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:18.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:18 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:19.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:19 smithi186 ceph-mon[104393]: pgmap v515: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 4.90k op/s 2023-10-08T10:02:19.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:19 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 224 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:19.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:19 smithi097 ceph-mon[99491]: pgmap v515: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 4.90k op/s 2023-10-08T10:02:19.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:19 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 224 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:19.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:19 smithi178 ceph-mon[103156]: pgmap v515: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 4.90k op/s 2023-10-08T10:02:19.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:19 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 224 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:21.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:21 smithi186 ceph-mon[104393]: pgmap v516: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.40k op/s 2023-10-08T10:02:21.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:21 smithi097 ceph-mon[99491]: pgmap v516: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.40k op/s 2023-10-08T10:02:21.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:21 smithi178 ceph-mon[103156]: pgmap v516: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.40k op/s 2023-10-08T10:02:22.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:22 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:22.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:22 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:22.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:22 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:22.956 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:22 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:23.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:22 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:23.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:22 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:23.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:23 smithi097 ceph-mon[99491]: pgmap v517: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.41k op/s 2023-10-08T10:02:23.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:23 smithi178 ceph-mon[103156]: pgmap v517: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.41k op/s 2023-10-08T10:02:24.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:23 smithi186 ceph-mon[104393]: pgmap v517: 129 pgs: 129 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 4.41k op/s 2023-10-08T10:02:25.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:25 smithi097 ceph-mon[99491]: pgmap v518: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-10-08T10:02:25.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:25 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 229 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:25.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:25 smithi178 ceph-mon[103156]: pgmap v518: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-10-08T10:02:25.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:25 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 229 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:26.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:25 smithi186 ceph-mon[104393]: pgmap v518: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-10-08T10:02:26.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:25 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 229 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:27.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:27 smithi097 ceph-mon[99491]: pgmap v519: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:02:27.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:27 smithi178 ceph-mon[103156]: pgmap v519: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:02:28.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:27 smithi186 ceph-mon[104393]: pgmap v519: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85837fe700 1 -- 172.21.15.97:0/2256286215 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f621f10ad0 con 0x55f621fc2680 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85837fe700 10 client.16203 handle_command_reply: tid=0 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 client.16203 shutdown 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 msgr2=0x55f621fe4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fe4cb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f85980073b0 tx=0x7f8598012ee0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 shutdown_connections 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f621fd72c0 0x55f621fe5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x55f621fc2680 0x55f621fc4aa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f621fd5e50 0x55f621fe4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85837fe700 0 client.16203 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 --2- 172.21.15.97:0/2256286215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f621fd67b0 0x55f621fe4cb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.747+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 >> 172.21.15.97:0/2256286215 conn(0x55f621f39200 msgr2=0x55f621fdedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 shutdown_connections 2023-10-08T10:02:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85b689cb80 1 -- 172.21.15.97:0/2256286215 wait complete. 2023-10-08T10:02:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85b689cb80 20 client.16203 trim_cache size 0 max 16384 2023-10-08T10:02:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T10:02:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T10:02:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T10:02:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T10:02:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 msgr2=0x7f858c0a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 0x7f858c0a3ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8594096330 tx=0x7f8594010850 comp rx=0 tx=0).stop 2023-10-08T10:02:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 msgr2=0x7f85a41a8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 0x7f85a41a8ad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f859400b3f0 tx=0x7f859400b700 comp rx=0 tx=0).stop 2023-10-08T10:02:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 shutdown_connections 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f858c0a1600 0x7f858c0a3ac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a41a66e0 0x7f85a41a8ad0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a4109dd0 0x7f85a41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 --2- 172.21.15.97:0/3198039455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85a4110f30 0x7f85a41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 >> 172.21.15.97:0/3198039455 conn(0x7f85a406d6e0 msgr2=0x7f85a4108970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 shutdown_connections 2023-10-08T10:02:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:28.748+0000 7f85aa6b3700 1 -- 172.21.15.97:0/3198039455 wait complete. 2023-10-08T10:02:29.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:29 smithi097 ceph-mon[99491]: pgmap v520: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-10-08T10:02:29.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:29 smithi097 ceph-mon[99491]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:29.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:29 smithi178 ceph-mon[103156]: pgmap v520: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-10-08T10:02:29.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:29 smithi178 ceph-mon[103156]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:30.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:29 smithi186 ceph-mon[104393]: pgmap v520: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-10-08T10:02:30.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:29 smithi186 ceph-mon[104393]: scrub summary: idle+waiting paths [/] 2023-10-08T10:02:30.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:30 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:30.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:30 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:30.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:30 smithi097 ceph-mon[99491]: Health check update: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:30.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:30 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:30.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:30 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:30.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:30 smithi178 ceph-mon[103156]: Health check update: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:31.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:30 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:31.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:30 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:31.026 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:30 smithi186 ceph-mon[104393]: Health check update: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops (SLOW_OPS) 2023-10-08T10:02:31.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:31 smithi097 ceph-mon[99491]: pgmap v521: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.44k op/s 2023-10-08T10:02:31.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:31 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T10:02:31.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:31 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T10:02:31.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:31 smithi097 ceph-mon[99491]: scrub summary: idle 2023-10-08T10:02:31.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:31 smithi178 ceph-mon[103156]: pgmap v521: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.44k op/s 2023-10-08T10:02:31.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:31 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T10:02:31.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:31 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T10:02:31.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:31 smithi178 ceph-mon[103156]: scrub summary: idle 2023-10-08T10:02:32.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:31 smithi186 ceph-mon[104393]: pgmap v521: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.44k op/s 2023-10-08T10:02:32.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:31 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T10:02:32.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:31 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T10:02:32.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:31 smithi186 ceph-mon[104393]: scrub summary: idle 2023-10-08T10:02:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:33 smithi097 ceph-mon[99491]: pgmap v522: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2023-10-08T10:02:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:33 smithi097 ceph-mon[99491]: MDS health message cleared (mds.?): 13 slow metadata IOs are blocked > 180 secs, oldest blocked for 235 secs 2023-10-08T10:02:33.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:33 smithi178 ceph-mon[103156]: pgmap v522: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2023-10-08T10:02:33.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:33 smithi178 ceph-mon[103156]: MDS health message cleared (mds.?): 13 slow metadata IOs are blocked > 180 secs, oldest blocked for 235 secs 2023-10-08T10:02:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:33 smithi186 ceph-mon[104393]: pgmap v522: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2023-10-08T10:02:34.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:33 smithi186 ceph-mon[104393]: MDS health message cleared (mds.?): 13 slow metadata IOs are blocked > 180 secs, oldest blocked for 235 secs 2023-10-08T10:02:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:34 smithi097 ceph-mon[99491]: Health check cleared: MDS_SLOW_METADATA_IO (was: 1 MDSs report slow metadata IOs) 2023-10-08T10:02:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:34 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:34 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:34.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:34 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:02:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:34 smithi178 ceph-mon[103156]: Health check cleared: MDS_SLOW_METADATA_IO (was: 1 MDSs report slow metadata IOs) 2023-10-08T10:02:34.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:34 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:34 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:34.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:34 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:02:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:34 smithi186 ceph-mon[104393]: Health check cleared: MDS_SLOW_METADATA_IO (was: 1 MDSs report slow metadata IOs) 2023-10-08T10:02:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:34 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] up:active 2023-10-08T10:02:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:34 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:02:35.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:34 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:02:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:35 smithi097 ceph-mon[99491]: pgmap v523: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2023-10-08T10:02:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:35 smithi097 ceph-mon[99491]: Health check cleared: SLOW_OPS (was: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops) 2023-10-08T10:02:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:35 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T10:02:35.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:35 smithi178 ceph-mon[103156]: pgmap v523: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2023-10-08T10:02:35.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:35 smithi178 ceph-mon[103156]: Health check cleared: SLOW_OPS (was: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops) 2023-10-08T10:02:35.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:35 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T10:02:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:35 smithi186 ceph-mon[104393]: pgmap v523: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2023-10-08T10:02:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:35 smithi186 ceph-mon[104393]: Health check cleared: SLOW_OPS (was: 0 slow ops, oldest one blocked for 236 sec, osd.9 has slow ops) 2023-10-08T10:02:36.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:35 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T10:02:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:37 smithi097 ceph-mon[99491]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.1 MiB/s wr, 1.78k op/s 2023-10-08T10:02:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:02:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:02:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:02:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:37 smithi178 ceph-mon[103156]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.1 MiB/s wr, 1.78k op/s 2023-10-08T10:02:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:02:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:02:37.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:02:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:37 smithi186 ceph-mon[104393]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.1 MiB/s wr, 1.78k op/s 2023-10-08T10:02:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:02:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:02:38.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:02:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:39 smithi097 ceph-mon[99491]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 1.54k op/s 2023-10-08T10:02:39.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:39 smithi178 ceph-mon[103156]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 1.54k op/s 2023-10-08T10:02:40.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:39 smithi186 ceph-mon[104393]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 1.54k op/s 2023-10-08T10:02:41.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:41 smithi097 ceph-mon[99491]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.1 MiB/s wr, 534 op/s 2023-10-08T10:02:41.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:41 smithi178 ceph-mon[103156]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.1 MiB/s wr, 534 op/s 2023-10-08T10:02:42.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:41 smithi186 ceph-mon[104393]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.1 MiB/s wr, 534 op/s 2023-10-08T10:02:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- 172.21.15.97:0/2285433776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 msgr2=0x7fbf600ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/2285433776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf600ffa60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbf500099c0 tx=0x7fbf50009cd0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- 172.21.15.97:0/2285433776 shutdown_connections 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/2285433776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601011d0 0x7fbf60107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/2285433776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf600ffa60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/2285433776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf60100830 0x7fbf60100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- 172.21.15.97:0/2285433776 >> 172.21.15.97:0/2285433776 conn(0x7fbf600fabb0 msgr2=0x7fbf600fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- 172.21.15.97:0/2285433776 shutdown_connections 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- 172.21.15.97:0/2285433776 wait complete. 2023-10-08T10:02:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 Processor -- start 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.319+0000 7fbf64b9f700 1 -- start start 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60100830 0x7fbf60196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 0x7fbf6019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6019e1e0 con 0x7fbf600ff640 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6019e320 con 0x7fbf60100830 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6019e4b0 con 0x7fbf601011d0 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58488/0 (socket says 172.21.15.97:58488) 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 -- 172.21.15.97:0/196316274 learned_addr learned my addr 172.21.15.97:0/196316274 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 0x7fbf6019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60100830 0x7fbf60196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 -- 172.21.15.97:0/196316274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 msgr2=0x7fbf6019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 0x7fbf6019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 -- 172.21.15.97:0/196316274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60100830 msgr2=0x7fbf60196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60100830 0x7fbf60196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 -- 172.21.15.97:0/196316274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500095f0 con 0x7fbf600ff640 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 0x7fbf6019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fd0 tx=0x7fbf50005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf577fe700 1 -- 172.21.15.97:0/196316274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5001d070 con 0x7fbf600ff640 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf577fe700 1 -- 172.21.15.97:0/196316274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf500059a0 con 0x7fbf600ff640 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6019e730 con 0x7fbf600ff640 2023-10-08T10:02:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.320+0000 7fbf577fe700 1 -- 172.21.15.97:0/196316274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50017650 con 0x7fbf600ff640 2023-10-08T10:02:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.321+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6019ebe0 con 0x7fbf600ff640 2023-10-08T10:02:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.323+0000 7fbf577fe700 1 -- 172.21.15.97:0/196316274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf50022470 con 0x7fbf600ff640 2023-10-08T10:02:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.323+0000 7fbf577fe700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 0x7fbf4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.323+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 0x7fbf4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.323+0000 7fbf577fe700 1 -- 172.21.15.97:0/196316274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf500ce570 con 0x7fbf600ff640 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 0x7fbf4c0a3940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005950 tx=0x7fbf480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 Processor -- start 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 -- start start 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b358dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358e2b0 0x5584b358e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b34f5400 con 0x5584b358ce40 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b34e6bc0 con 0x5584b358d7a0 2023-10-08T10:02:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.324+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b34e44e0 con 0x5584b358e2b0 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58496/0 (socket says 172.21.15.97:58496) 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 -- 172.21.15.97:0/328192972 learned_addr learned my addr 172.21.15.97:0/328192972 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358e2b0 0x5584b358e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b358dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 -- 172.21.15.97:0/328192972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358e2b0 msgr2=0x5584b358e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358e2b0 0x5584b358e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 -- 172.21.15.97:0/328192972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 msgr2=0x5584b358dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b358dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 -- 172.21.15.97:0/328192972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b330ad80 con 0x5584b358ce40 2023-10-08T10:02:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009990 tx=0x7fbf50022b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173dff1271728c57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf54ff9700 1 -- 172.21.15.97:0/328192972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5009be10 con 0x5584b358ce40 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf54ff9700 1 -- 172.21.15.97:0/328192972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf500b5070 con 0x5584b358ce40 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.325+0000 7fbf54ff9700 1 -- 172.21.15.97:0/328192972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50099ab0 con 0x5584b358ce40 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 -- 172.21.15.97:0/328192972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 msgr2=0x5584b358d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009990 tx=0x7fbf50022b10 comp rx=0 tx=0).stop 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 -- 172.21.15.97:0/328192972 shutdown_connections 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358e2b0 0x5584b358e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358ce40 0x5584b358d260 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/328192972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b358dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 -- 172.21.15.97:0/328192972 >> 172.21.15.97:0/328192972 conn(0x5584b3501200 msgr2=0x5584b35878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 -- 172.21.15.97:0/328192972 shutdown_connections 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.326+0000 7fbf70d88b80 1 -- 172.21.15.97:0/328192972 wait complete. 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 -- start start 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 Processor -- start 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b3596580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358e2b0 0x5584b359dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b34dfdd0 con 0x5584b358d7a0 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b347c6b0 con 0x5584b358e2b0 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b347c350 con 0x5584b358ce40 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf70d88b80 10 client.0 _ll_register_callbacks cb 0x5584b32d97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58504/0 (socket says 172.21.15.97:58504) 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b3596580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 -- 172.21.15.97:0/3376228506 learned_addr learned my addr 172.21.15.97:0/3376228506 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358e2b0 0x5584b359dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 -- 172.21.15.97:0/3376228506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 msgr2=0x5584b3596580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b3596580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 -- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358e2b0 msgr2=0x5584b359dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358e2b0 0x5584b359dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.327+0000 7fbf5dd9b700 1 -- 172.21.15.97:0/3376228506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6019ebe0 con 0x5584b358d7a0 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b3596580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbf48011720 tx=0x7fbf48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7611b1942efe13c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:02:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48027600 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf4801f3f0 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584b34a7e50 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf48027760 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf70d88b80 10 client.16227 fetch_fsmap learned FSMap version 49 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5584b32f3c00 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584b34a7e50 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.328+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf48025020 con 0x5584b358d7a0 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 20 client.16227 populate_metadata read hostname 'smithi097' 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35a23c0 0x5584b35a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 4 client.16227 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584b34d8db0 con 0x5584b35a23c0 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35a23c0 0x5584b35a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf45ffb700 10 client.16227 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.329+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35a23c0 0x5584b35a47e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e4345bfcc4b9b server_cookie=339a84b47c9371e8 in_seq=0 out_seq=0 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf45ffb700 1 -- 172.21.15.97:0/3376228506 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5584b34d8db0 con 0x5584b35a23c0 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf45ffb700 10 client.16227 handle_command_reply: tid=0 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 client.16227 shutdown 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 msgr2=0x5584b3596ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbf48011720 tx=0x7fbf48004750 comp rx=0 tx=0).stop 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 shutdown_connections 2023-10-08T10:02:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358e2b0 0x5584b359dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35a23c0 0x5584b35a47e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b3596580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/3376228506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358d7a0 0x5584b3596ac0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf45ffb700 0 client.16227 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 >> 172.21.15.97:0/3376228506 conn(0x5584b3501200 msgr2=0x5584b3587010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 shutdown_connections 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 1 -- 172.21.15.97:0/3376228506 wait complete. 2023-10-08T10:02:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.330+0000 7fbf70d88b80 20 client.16227 trim_cache size 0 max 16384 2023-10-08T10:02:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.345+0000 7fbf70d88b80 1 Processor -- start 2023-10-08T10:02:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.345+0000 7fbf70d88b80 1 -- start start 2023-10-08T10:02:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358ce40 0x5584b3595a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358d7a0 0x5584b359da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b33608c0 con 0x5584b358e2b0 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b35896e0 con 0x5584b358ce40 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b34f5400 con 0x5584b358d7a0 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58514/0 (socket says 172.21.15.97:58514) 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/2175233024 learned_addr learned my addr 172.21.15.97:0/2175233024 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358d7a0 0x5584b359da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358ce40 0x5584b3595a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/2175233024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358d7a0 msgr2=0x5584b359da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358d7a0 0x5584b359da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/2175233024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358ce40 msgr2=0x5584b3595a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358ce40 0x5584b3595a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/2175233024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b33311f0 con 0x5584b358e2b0 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbf580076c0 tx=0x7fbf5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85026dbb30dcbd08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf557fa700 1 -- 172.21.15.97:0/2175233024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800e040 con 0x5584b358e2b0 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.346+0000 7fbf557fa700 1 -- 172.21.15.97:0/2175233024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5800f040 con 0x5584b358e2b0 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf557fa700 1 -- 172.21.15.97:0/2175233024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58015cb0 con 0x5584b358e2b0 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- 172.21.15.97:0/2175233024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 msgr2=0x5584b358e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbf580076c0 tx=0x7fbf5800b040 comp rx=0 tx=0).stop 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- 172.21.15.97:0/2175233024 shutdown_connections 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b358e690 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358ce40 0x5584b3595a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/2175233024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358d7a0 0x5584b359da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- 172.21.15.97:0/2175233024 >> 172.21.15.97:0/2175233024 conn(0x5584b3501200 msgr2=0x5584b35a9140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- 172.21.15.97:0/2175233024 shutdown_connections 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- 172.21.15.97:0/2175233024 wait complete. 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 -- start start 2023-10-08T10:02:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.347+0000 7fbf70d88b80 1 Processor -- start 2023-10-08T10:02:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b359b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b359be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b359d310 con 0x5584b358e2b0 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b359d480 con 0x5584b358d7a0 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b359d640 con 0x5584b358ce40 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf70d88b80 10 client.0 _ll_register_callbacks cb 0x7fbf4800ee30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58520/0 (socket says 172.21.15.97:58520) 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/4088709796 learned_addr learned my addr 172.21.15.97:0/4088709796 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b359b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b359be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/4088709796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 msgr2=0x5584b359b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b359b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 msgr2=0x5584b359be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b359be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 -- 172.21.15.97:0/4088709796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf58009360 con 0x5584b358e2b0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5dd9b700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b359be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf5ed9d700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbf580094f0 tx=0x7fbf58009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234eade7f7bca494 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.348+0000 7fbf45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800e070 con 0x5584b358e2b0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf58017620 con 0x5584b358e2b0 2023-10-08T10:02:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf500b5070 con 0x5584b358e2b0 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf58025070 con 0x5584b358e2b0 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 10 client.16233 fetch_fsmap learned FSMap version 49 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5584b3309250 con 0x5584b358e2b0 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf500b5070 con 0x5584b358e2b0 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf580207e0 con 0x5584b358e2b0 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 20 client.16233 populate_metadata read hostname 'smithi097' 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35f7190 0x5584b35f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 4 client.16233 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584b34d8ad0 con 0x5584b35f7190 2023-10-08T10:02:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.349+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35f7190 0x5584b35f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.350+0000 7fbf45ffb700 10 client.16233 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.350+0000 7fbf5e59c700 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35f7190 0x5584b35f95b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c223f32c87ce23 server_cookie=a3e0dff5719e12c1 in_seq=0 out_seq=0 2023-10-08T10:02:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf45ffb700 1 -- 172.21.15.97:0/4088709796 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584b34d8ad0 con 0x5584b35f7190 2023-10-08T10:02:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf45ffb700 10 client.16233 handle_command_reply: tid=0 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 client.16233 shutdown 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 msgr2=0x5584b359cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbf580094f0 tx=0x7fbf58009920 comp rx=0 tx=0).stop 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 shutdown_connections 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584b358e2b0 0x5584b359cdd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5584b35f7190 0x5584b35f95b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584b358ce40 0x5584b359b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 --2- 172.21.15.97:0/4088709796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5584b358d7a0 0x5584b359be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf45ffb700 0 client.16233 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 >> 172.21.15.97:0/4088709796 conn(0x5584b3501200 msgr2=0x5584b35947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 shutdown_connections 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 1 -- 172.21.15.97:0/4088709796 wait complete. 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.448+0000 7fbf70d88b80 20 client.16233 trim_cache size 0 max 16384 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T10:02:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T10:02:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 msgr2=0x7fbf4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 0x7fbf4c0a3940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005950 tx=0x7fbf480058e0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 msgr2=0x7fbf601966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fd0 tx=0x7fbf50005f10 comp rx=0 tx=0).stop 2023-10-08T10:02:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 shutdown_connections 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fbf4c0a1480 0x7fbf4c0a3940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf601011d0 0x7fbf6019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf600ff640 0x7fbf601966b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 --2- 172.21.15.97:0/196316274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60100830 0x7fbf60196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 >> 172.21.15.97:0/196316274 conn(0x7fbf600fabb0 msgr2=0x7fbf600fb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 shutdown_connections 2023-10-08T10:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:42.449+0000 7fbf64b9f700 1 -- 172.21.15.97:0/196316274 wait complete. 2023-10-08T10:02:43.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:43 smithi097 ceph-mon[99491]: pgmap v527: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 890 KiB/s wr, 235 op/s 2023-10-08T10:02:43.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:43 smithi178 ceph-mon[103156]: pgmap v527: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 890 KiB/s wr, 235 op/s 2023-10-08T10:02:44.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:43 smithi186 ceph-mon[104393]: pgmap v527: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 890 KiB/s wr, 235 op/s 2023-10-08T10:02:45.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:45 smithi097 ceph-mon[99491]: pgmap v528: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 367 KiB/s wr, 109 op/s 2023-10-08T10:02:45.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:45 smithi178 ceph-mon[103156]: pgmap v528: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 367 KiB/s wr, 109 op/s 2023-10-08T10:02:46.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:45 smithi186 ceph-mon[104393]: pgmap v528: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 367 KiB/s wr, 109 op/s 2023-10-08T10:02:47.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:47 smithi097 ceph-mon[99491]: pgmap v529: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2023-10-08T10:02:47.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:47 smithi178 ceph-mon[103156]: pgmap v529: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2023-10-08T10:02:48.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:47 smithi186 ceph-mon[104393]: pgmap v529: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2023-10-08T10:02:49.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:49 smithi097 ceph-mon[99491]: pgmap v530: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2023-10-08T10:02:49.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:49 smithi178 ceph-mon[103156]: pgmap v530: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2023-10-08T10:02:50.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:49 smithi186 ceph-mon[104393]: pgmap v530: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2023-10-08T10:02:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:51 smithi097 ceph-mon[99491]: pgmap v531: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-10-08T10:02:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:51 smithi178 ceph-mon[103156]: pgmap v531: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-10-08T10:02:52.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:51 smithi186 ceph-mon[104393]: pgmap v531: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-10-08T10:02:53.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:53 smithi097 ceph-mon[99491]: pgmap v532: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2023-10-08T10:02:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:53 smithi178 ceph-mon[103156]: pgmap v532: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2023-10-08T10:02:54.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:53 smithi186 ceph-mon[104393]: pgmap v532: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2023-10-08T10:02:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 -- 172.21.15.97:0/2180798056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec1002b0 msgr2=0x7f5cec107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2180798056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec1002b0 0x7f5cec107b70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400b0a0 tx=0x7f5ce400b3b0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 -- 172.21.15.97:0/2180798056 shutdown_connections 2023-10-08T10:02:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2180798056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec1002b0 0x7f5cec107b70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2180798056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2180798056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec0ff910 0x7f5cec0ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 -- 172.21.15.97:0/2180798056 >> 172.21.15.97:0/2180798056 conn(0x7f5cec0fa340 msgr2=0x7f5cec0fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 -- 172.21.15.97:0/2180798056 shutdown_connections 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.561+0000 7f5cf3223700 1 -- 172.21.15.97:0/2180798056 wait complete. 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 Processor -- start 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 -- start start 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec1002b0 0x7f5cec101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1029e0 con 0x7f5cec0ff910 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec102b50 con 0x7f5cec1002b0 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf3223700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a07f0 con 0x7f5cec100be0 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58540/0 (socket says 172.21.15.97:58540) 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/2507632998 learned_addr learned my addr 172.21.15.97:0/2507632998 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cebfff700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec1002b0 0x7f5cec101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.562+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/2507632998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 msgr2=0x7f5cec102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/2507632998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec1002b0 msgr2=0x7f5cec101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec1002b0 0x7f5cec101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/2507632998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009d00 con 0x7f5cec0ff910 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c970 tx=0x7f5ce000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5ce9ffb700 1 -- 172.21.15.97:0/2507632998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0004220 con 0x7f5cec0ff910 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5ce9ffb700 1 -- 172.21.15.97:0/2507632998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce000ce60 con 0x7f5cec0ff910 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5ce9ffb700 1 -- 172.21.15.97:0/2507632998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0003c00 con 0x7f5cec0ff910 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1a0a80 con 0x7f5cec0ff910 2023-10-08T10:02:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.563+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1a0f40 con 0x7f5cec0ff910 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.565+0000 7f5ce9ffb700 1 -- 172.21.15.97:0/2507632998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5ce0004380 con 0x7f5cec0ff910 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.566+0000 7f5ce9ffb700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 0x7f5cdc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.566+0000 7f5ce9ffb700 1 -- 172.21.15.97:0/2507632998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5ce00ccca0 con 0x7f5cec0ff910 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.566+0000 7f5cebfff700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 0x7f5cdc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.566+0000 7f5cebfff700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 0x7f5cdc0a39c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005950 tx=0x7f5cd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 Processor -- start 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 -- start start 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d2e40 0x5611989d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56119893b400 con 0x5611989d42b0 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56119892cbc0 con 0x5611989d37a0 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56119892a4e0 con 0x5611989d2e40 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58552/0 (socket says 172.21.15.97:58552) 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 -- 172.21.15.97:0/1611008735 learned_addr learned my addr 172.21.15.97:0/1611008735 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d2e40 0x5611989d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cebfff700 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 -- 172.21.15.97:0/1611008735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d2e40 msgr2=0x5611989d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d2e40 0x5611989d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 -- 172.21.15.97:0/1611008735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 msgr2=0x5611989d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.567+0000 7f5cf17c0700 1 -- 172.21.15.97:0/1611008735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561198750d80 con 0x5611989d42b0 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40062a0 tx=0x7f5ce4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8412673398238267 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cd6ffd700 1 -- 172.21.15.97:0/1611008735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400e050 con 0x5611989d42b0 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cd6ffd700 1 -- 172.21.15.97:0/1611008735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce40090d0 con 0x5611989d42b0 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cd6ffd700 1 -- 172.21.15.97:0/1611008735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40052a0 con 0x5611989d42b0 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cff40cb80 1 -- 172.21.15.97:0/1611008735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 msgr2=0x5611989d4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40062a0 tx=0x7f5ce4009c20 comp rx=0 tx=0).stop 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cff40cb80 1 -- 172.21.15.97:0/1611008735 shutdown_connections 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.568+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989d4730 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d2e40 0x5611989d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/1611008735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 -- 172.21.15.97:0/1611008735 >> 172.21.15.97:0/1611008735 conn(0x561198947200 msgr2=0x5611989cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 -- 172.21.15.97:0/1611008735 shutdown_connections 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 -- 172.21.15.97:0/1611008735 wait complete. 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 -- start start 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.569+0000 7f5cff40cb80 1 Processor -- start 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d42b0 0x5611989e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561198925dd0 con 0x5611989d2e40 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611988c26b0 con 0x5611989d37a0 2023-10-08T10:02:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611988c2350 con 0x5611989d42b0 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 10 client.0 _ll_register_callbacks cb 0x56119871f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58554/0 (socket says 172.21.15.97:58554) 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/4202688555 learned_addr learned my addr 172.21.15.97:0/4202688555 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cebfff700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d42b0 0x5611989e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/4202688555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d42b0 msgr2=0x5611989e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d42b0 0x5611989e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 msgr2=0x5611989dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989dcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/4202688555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1083e0 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c940 tx=0x7f5ce009a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327b52dbb30cfd07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5ccbfff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00b9050 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5ce0099120 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611988ede50 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ce00992e0 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 10 client.16245 fetch_fsmap learned FSMap version 49 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.570+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561198739c00 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce0063020 con 0x5611989d2e40 2023-10-08T10:02:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5ce00b2200 con 0x5611989d2e40 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 20 client.16245 populate_metadata read hostname 'smithi097' 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5611989e8450 0x5611989ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 4 client.16245 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56119891edb0 con 0x5611989e8450 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.571+0000 7f5cebfff700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5611989e8450 0x5611989ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cebfff700 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5611989e8450 0x5611989ea840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b754b6a35b9bf8 server_cookie=a747cd4098cc0eec in_seq=0 out_seq=0 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5ccbfff700 10 client.16245 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5ccbfff700 1 -- 172.21.15.97:0/4202688555 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56119891edb0 con 0x5611989e8450 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5ccbfff700 10 client.16245 handle_command_reply: tid=0 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 client.16245 shutdown 2023-10-08T10:02:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 msgr2=0x5611989dc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c940 tx=0x7f5ce009a470 comp rx=0 tx=0).stop 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 shutdown_connections 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5611989e8450 0x5611989ea840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d42b0 0x5611989e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989dc880 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5ccbfff700 0 client.16245 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/4202688555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d37a0 0x5611989dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 >> 172.21.15.97:0/4202688555 conn(0x561198947200 msgr2=0x5611989cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 shutdown_connections 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.572+0000 7f5cff40cb80 1 -- 172.21.15.97:0/4202688555 wait complete. 2023-10-08T10:02:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.573+0000 7f5cff40cb80 20 client.16245 trim_cache size 0 max 16384 2023-10-08T10:02:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 Processor -- start 2023-10-08T10:02:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 -- start start 2023-10-08T10:02:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 0x5611989da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611989cf390 con 0x5611989d2e40 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987a68c0 con 0x5611989d42b0 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611989ca850 con 0x5611989d37a0 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cf0fbf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58562/0 (socket says 172.21.15.97:58562) 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.587+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/804222062 learned_addr learned my addr 172.21.15.97:0/804222062 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 0x5611989da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cebfff700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/804222062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 msgr2=0x5611989e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/804222062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 msgr2=0x5611989da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 0x5611989da840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 -- 172.21.15.97:0/804222062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611987771f0 con 0x5611989d2e40 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 0x5611989da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000cfd0 tx=0x7f5ce000a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f212f725ae0a65d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cd77fe700 1 -- 172.21.15.97:0/804222062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00b9070 con 0x5611989d2e40 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cd77fe700 1 -- 172.21.15.97:0/804222062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce0063030 con 0x5611989d2e40 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.588+0000 7f5cd77fe700 1 -- 172.21.15.97:0/804222062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0065510 con 0x5611989d2e40 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- 172.21.15.97:0/804222062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 msgr2=0x5611989d4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000cfd0 tx=0x7f5ce000a610 comp rx=0 tx=0).stop 2023-10-08T10:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- 172.21.15.97:0/804222062 shutdown_connections 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d42b0 0x5611989da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d2e40 0x5611989d4690 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/804222062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- 172.21.15.97:0/804222062 >> 172.21.15.97:0/804222062 conn(0x561198947200 msgr2=0x5611989cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- 172.21.15.97:0/804222062 shutdown_connections 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- 172.21.15.97:0/804222062 wait complete. 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- start start 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 Processor -- start 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 0x5611989e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611989e3540 con 0x5611989d42b0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611989e36b0 con 0x5611989d2e40 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611989e3870 con 0x5611989d37a0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cff40cb80 10 client.0 _ll_register_callbacks cb 0x7f5ce009a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.589+0000 7f5cf17c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58572/0 (socket says 172.21.15.97:58572) 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 -- 172.21.15.97:0/2114469817 learned_addr learned my addr 172.21.15.97:0/2114469817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 0x5611989e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cebfff700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 -- 172.21.15.97:0/2114469817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 msgr2=0x5611989e2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 -- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 msgr2=0x5611989e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:02:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 0x5611989e1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 -- 172.21.15.97:0/2114469817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561198739c00 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 0x5611989e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cf17c0700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4000f80 tx=0x7f5ce4008fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb0656ee2e6e948 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5ccbfff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40078e0 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5ce4004c50 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ce40090d0 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ce400e030 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cff40cb80 10 client.16251 fetch_fsmap learned FSMap version 49 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56119874f250 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.590+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce4037020 con 0x5611989d42b0 2023-10-08T10:02:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5ce400b0d0 con 0x5611989d42b0 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 20 client.16251 populate_metadata read hostname 'smithi097' 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561198a3cea0 0x561198a3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 4 client.16251 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56119891ead0 con 0x561198a3cea0 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561198a3cea0 0x561198a3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5ccbfff700 10 client.16251 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.591+0000 7f5cf0fbf700 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561198a3cea0 0x561198a3f290 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4064c731af113175 server_cookie=1b6dd2a420896ed7 in_seq=0 out_seq=0 2023-10-08T10:02:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5ccbfff700 1 -- 172.21.15.97:0/2114469817 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56119891ead0 con 0x561198a3cea0 2023-10-08T10:02:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5ccbfff700 10 client.16251 handle_command_reply: tid=0 2023-10-08T10:02:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 client.16251 shutdown 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 msgr2=0x5611989e3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4000f80 tx=0x7f5ce4008fc0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 shutdown_connections 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611989d42b0 0x5611989e3000 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5611989d2e40 0x5611989e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x561198a3cea0 0x561198a3f290 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 --2- 172.21.15.97:0/2114469817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5611989d37a0 0x5611989e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5ccbfff700 0 client.16251 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 >> 172.21.15.97:0/2114469817 conn(0x561198947200 msgr2=0x5611989dc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 shutdown_connections 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 1 -- 172.21.15.97:0/2114469817 wait complete. 2023-10-08T10:02:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.596+0000 7f5cff40cb80 20 client.16251 trim_cache size 0 max 16384 2023-10-08T10:02:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T10:02:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T10:02:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T10:02:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 msgr2=0x7f5cdc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 0x7f5cdc0a39c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005950 tx=0x7f5cd800b410 comp rx=0 tx=0).stop 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 msgr2=0x7f5cec103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c970 tx=0x7f5ce000cc80 comp rx=0 tx=0).stop 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 shutdown_connections 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5cdc0a1500 0x7f5cdc0a39c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cec100be0 0x7f5cec102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cec0ff910 0x7f5cec103670 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 --2- 172.21.15.97:0/2507632998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cec1002b0 0x7f5cec101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:02:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 >> 172.21.15.97:0/2507632998 conn(0x7f5cec0fa340 msgr2=0x7f5cec0fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:02:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 shutdown_connections 2023-10-08T10:02:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:02:55.597+0000 7f5cf3223700 1 -- 172.21.15.97:0/2507632998 wait complete. 2023-10-08T10:02:55.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:55 smithi097 ceph-mon[99491]: pgmap v533: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2023-10-08T10:02:55.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:55 smithi178 ceph-mon[103156]: pgmap v533: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2023-10-08T10:02:56.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:55 smithi186 ceph-mon[104393]: pgmap v533: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2023-10-08T10:02:57.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:57 smithi097 ceph-mon[99491]: pgmap v534: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-08T10:02:57.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:57 smithi178 ceph-mon[103156]: pgmap v534: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-08T10:02:58.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:57 smithi186 ceph-mon[104393]: pgmap v534: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-08T10:02:59.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:02:59 smithi097 ceph-mon[99491]: pgmap v535: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-10-08T10:02:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:02:59 smithi178 ceph-mon[103156]: pgmap v535: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-10-08T10:03:00.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:02:59 smithi186 ceph-mon[104393]: pgmap v535: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-10-08T10:03:01.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:01 smithi097 ceph-mon[99491]: pgmap v536: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-10-08T10:03:01.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:01 smithi178 ceph-mon[103156]: pgmap v536: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-10-08T10:03:02.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:01 smithi186 ceph-mon[104393]: pgmap v536: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-10-08T10:03:03.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:03 smithi097 ceph-mon[99491]: pgmap v537: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-08T10:03:03.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:03 smithi178 ceph-mon[103156]: pgmap v537: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-08T10:03:04.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:03 smithi186 ceph-mon[104393]: pgmap v537: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-08T10:03:05.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:05 smithi097 ceph-mon[99491]: pgmap v538: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-08T10:03:05.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:05 smithi178 ceph-mon[103156]: pgmap v538: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-08T10:03:06.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:05 smithi186 ceph-mon[104393]: pgmap v538: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-08T10:03:07.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:07 smithi097 ceph-mon[99491]: pgmap v539: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-10-08T10:03:07.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:07 smithi178 ceph-mon[103156]: pgmap v539: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-10-08T10:03:08.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:07 smithi186 ceph-mon[104393]: pgmap v539: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2496706279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 msgr2=0x7fdd3c100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2496706279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c100c90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdd240099c0 tx=0x7fdd24009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2496706279 shutdown_connections 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2496706279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 0x7fdd3c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2496706279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2496706279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c100c90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2496706279 >> 172.21.15.97:0/2496706279 conn(0x7fdd3c0fabb0 msgr2=0x7fdd3c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2496706279 shutdown_connections 2023-10-08T10:03:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.716+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2496706279 wait complete. 2023-10-08T10:03:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 Processor -- start 2023-10-08T10:03:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 -- start start 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 0x7fdd3c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19e150 con 0x7fdd3c100830 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19e2c0 con 0x7fdd3c0ff640 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.717+0000 7fdd42a5a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19e480 con 0x7fdd3c1011d0 2023-10-08T10:03:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58584/0 (socket says 172.21.15.97:58584) 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/2224959898 learned_addr learned my addr 172.21.15.97:0/2224959898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 0x7fdd3c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2224959898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 msgr2=0x7fdd3c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 0x7fdd3c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 msgr2=0x7fdd3c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2224959898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240095f0 con 0x7fdd3c0ff640 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0685e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00d740 tx=0x7fdd2c00da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd397fa700 1 -- 172.21.15.97:0/2224959898 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c004220 con 0x7fdd3c0ff640 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c19e760 con 0x7fdd3c0ff640 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd397fa700 1 -- 172.21.15.97:0/2224959898 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2c004380 con 0x7fdd3c0ff640 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd397fa700 1 -- 172.21.15.97:0/2224959898 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c003cf0 con 0x7fdd3c0ff640 2023-10-08T10:03:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.718+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c19ecc0 con 0x7fdd3c0ff640 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.721+0000 7fdd397fa700 1 -- 172.21.15.97:0/2224959898 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdd2c00f690 con 0x7fdd3c0ff640 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.721+0000 7fdd397fa700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 0x7fdd280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.721+0000 7fdd397fa700 1 -- 172.21.15.97:0/2224959898 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd2c0ccd00 con 0x7fdd3c0ff640 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.721+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 0x7fdd280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 0x7fdd280a3940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c068ba0 tx=0x7fdd24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 Processor -- start 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 -- start start 2023-10-08T10:03:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc1e40 0x5629dadc2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 0x5629dadc3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dad2a400 con 0x5629dadc27a0 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dad1bbc0 con 0x5629dadc32b0 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dad194e0 con 0x5629dadc1e40 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58592/0 (socket says 172.21.15.97:58592) 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.722+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/2168318246 learned_addr learned my addr 172.21.15.97:0/2168318246 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc1e40 0x5629dadc2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 0x5629dadc3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/2168318246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc1e40 msgr2=0x5629dadc2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc1e40 0x5629dadc2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/2168318246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 msgr2=0x5629dadc3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 0x5629dadc3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/2168318246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629dab3fd80 con 0x5629dadc27a0 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 0x5629dadc3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdd240095c0 tx=0x7fdd24020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d21d4d39bb21bf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd327fc700 1 -- 172.21.15.97:0/2168318246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400bea0 con 0x5629dadc27a0 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd327fc700 1 -- 172.21.15.97:0/2168318246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2401e040 con 0x5629dadc27a0 2023-10-08T10:03:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.723+0000 7fdd327fc700 1 -- 172.21.15.97:0/2168318246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004880 con 0x5629dadc27a0 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2168318246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 msgr2=0x5629dadc2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdd240095c0 tx=0x7fdd24020000 comp rx=0 tx=0).stop 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2168318246 shutdown_connections 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc32b0 0x5629dadc3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc1e40 0x5629dadc2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2168318246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadc2cd0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2168318246 >> 172.21.15.97:0/2168318246 conn(0x5629dad36200 msgr2=0x5629dadbc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2168318246 shutdown_connections 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2168318246 wait complete. 2023-10-08T10:03:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 -- start start 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.724+0000 7fdd4ec43b80 1 Processor -- start 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 0x5629dadcbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadd2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dad14dd0 con 0x5629dadc1e40 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dacb16b0 con 0x5629dadc27a0 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dacb1350 con 0x5629dadc32b0 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd4ec43b80 10 client.0 _ll_register_callbacks cb 0x5629dab0e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58596/0 (socket says 172.21.15.97:58596) 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2865056933 learned_addr learned my addr 172.21.15.97:0/2865056933 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadd2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 0x5629dadcbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2865056933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 msgr2=0x5629dadd2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadd2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 msgr2=0x5629dadcbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 0x5629dadcbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 -- 172.21.15.97:0/2865056933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c1083d0 con 0x5629dadc1e40 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 0x5629dadcbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5572c7bf6f661a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.725+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c09a150 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629dacdce50 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd2c099cf0 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd2c09de40 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 10 client.16263 fetch_fsmap learned FSMap version 49 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5629dab28c00 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2c063080 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdd2c0a1c80 con 0x5629dadc1e40 2023-10-08T10:03:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 20 client.16263 populate_metadata read hostname 'smithi097' 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dadd74e0 0x5629dadd9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 4 client.16263 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629dad0ddb0 con 0x5629dadd74e0 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.726+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dadd74e0 0x5629dadd9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.727+0000 7fdd1b7fe700 10 client.16263 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.727+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dadd74e0 0x5629dadd9900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feab9082688e6496 server_cookie=924b23e13d17254a in_seq=0 out_seq=0 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.727+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/2865056933 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629dad0ddb0 con 0x5629dadd74e0 2023-10-08T10:03:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.727+0000 7fdd1b7fe700 10 client.16263 handle_command_reply: tid=0 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.727+0000 7fdd4ec43b80 1 client.16263 shutdown 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 msgr2=0x5629dadcb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c003ed0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 shutdown_connections 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dadd74e0 0x5629dadd9900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadd2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd1b7fe700 0 client.16263 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc1e40 0x5629dadcb820 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/2865056933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc27a0 0x5629dadcbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 >> 172.21.15.97:0/2865056933 conn(0x5629dad36200 msgr2=0x5629dadbc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 shutdown_connections 2023-10-08T10:03:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/2865056933 wait complete. 2023-10-08T10:03:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.728+0000 7fdd4ec43b80 20 client.16263 trim_cache size 0 max 16384 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 Processor -- start 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 -- start start 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dadd33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadc98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dab958c0 con 0x5629dadc27a0 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dadd30b0 con 0x5629dadc1e40 2023-10-08T10:03:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dadc6e40 con 0x5629dadc32b0 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58606/0 (socket says 172.21.15.97:58606) 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.742+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/3553914041 learned_addr learned my addr 172.21.15.97:0/3553914041 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dadd33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadc98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/3553914041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 msgr2=0x5629dadc98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadc98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/3553914041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 msgr2=0x5629dadd33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dadd33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 -- 172.21.15.97:0/3553914041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629dab661f0 con 0x5629dadc27a0 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadc98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dadd33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009990 tx=0x7fdd2401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fe5328ae16412 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd32ffd700 1 -- 172.21.15.97:0/3553914041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2403ed40 con 0x5629dadc27a0 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd32ffd700 1 -- 172.21.15.97:0/3553914041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2403e070 con 0x5629dadc27a0 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.743+0000 7fdd32ffd700 1 -- 172.21.15.97:0/3553914041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24003e40 con 0x5629dadc27a0 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/3553914041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 msgr2=0x5629dadcad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009990 tx=0x7fdd2401a040 comp rx=0 tx=0).stop 2023-10-08T10:03:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/3553914041 shutdown_connections 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc32b0 0x5629dadc98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dadd33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/3553914041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc27a0 0x5629dadcad30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/3553914041 >> 172.21.15.97:0/3553914041 conn(0x5629dad36200 msgr2=0x5629dadde260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/3553914041 shutdown_connections 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/3553914041 wait complete. 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 -- start start 2023-10-08T10:03:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.744+0000 7fdd4ec43b80 1 Processor -- start 2023-10-08T10:03:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dae26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc27a0 0x5629dae272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dae2e820 con 0x5629dadc32b0 2023-10-08T10:03:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dae2e960 con 0x5629dadc1e40 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dae2eaf0 con 0x5629dadc27a0 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 10 client.0 _ll_register_callbacks cb 0x7fdd2c099830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58614/0 (socket says 172.21.15.97:58614) 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 -- 172.21.15.97:0/912214594 learned_addr learned my addr 172.21.15.97:0/912214594 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dae26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd3b7fe700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc27a0 0x5629dae272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 -- 172.21.15.97:0/912214594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc27a0 msgr2=0x5629dae272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc27a0 0x5629dae272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 -- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 msgr2=0x5629dae26d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dae26d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 -- 172.21.15.97:0/912214594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629dab3fd80 con 0x5629dadc32b0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd40ff7700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdd34007ba0 tx=0x7fdd34007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468539f60f2cf20b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340177b0 con 0x5629dadc32b0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd34017910 con 0x5629dadc32b0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.745+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd2401e040 con 0x5629dadc32b0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd34017ad0 con 0x5629dadc32b0 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 10 client.16269 fetch_fsmap learned FSMap version 49 2023-10-08T10:03:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5629dab3e250 con 0x5629dadc32b0 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2401e040 con 0x5629dadc32b0 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdd34018420 con 0x5629dadc32b0 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 20 client.16269 populate_metadata read hostname 'smithi097' 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dae306b0 0x5629dae32ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 4 client.16269 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 --> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629dad0dad0 con 0x5629dae306b0 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.746+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dae306b0 0x5629dae32ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.747+0000 7fdd1b7fe700 10 client.16269 ms_handle_connect on v2:172.21.15.178:6838/1665352653 2023-10-08T10:03:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.747+0000 7fdd3bfff700 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dae306b0 0x5629dae32ad0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b05854bc2ced9c0 server_cookie=b50c90fe43d6ebf8 in_seq=0 out_seq=0 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.755+0000 7fdd1b7fe700 1 -- 172.21.15.97:0/912214594 <== mds.0 v2:172.21.15.178:6838/1665352653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629dad0dad0 con 0x5629dae306b0 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.755+0000 7fdd1b7fe700 10 client.16269 handle_command_reply: tid=0 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.755+0000 7fdd4ec43b80 1 client.16269 shutdown 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 msgr2=0x5629dae2e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdd34007ba0 tx=0x7fdd34007bd0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 shutdown_connections 2023-10-08T10:03:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dadc32b0 0x5629dae2e2e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629dadc1e40 0x5629dae26d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.178:6838/1665352653,v1:172.21.15.178:6839/1665352653] conn(0x5629dae306b0 0x5629dae32ad0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 --2- 172.21.15.97:0/912214594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5629dadc27a0 0x5629dae272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd1b7fe700 0 client.16269 ms_handle_reset on v2:172.21.15.178:6838/1665352653 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 >> 172.21.15.97:0/912214594 conn(0x5629dad36200 msgr2=0x5629dadc9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 shutdown_connections 2023-10-08T10:03:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 1 -- 172.21.15.97:0/912214594 wait complete. 2023-10-08T10:03:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd4ec43b80 20 client.16269 trim_cache size 0 max 16384 2023-10-08T10:03:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2023-10-08T10:03:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2023-10-08T10:03:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2023-10-08T10:03:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2023-10-08T10:03:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 msgr2=0x7fdd280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 0x7fdd280a3940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c068ba0 tx=0x7fdd24005fb0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 msgr2=0x7fdd3c0685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0685e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00d740 tx=0x7fdd2c00da50 comp rx=0 tx=0).stop 2023-10-08T10:03:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 shutdown_connections 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fdd280a1480 0x7fdd280a3940 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd3c1011d0 0x7fdd3c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3c0ff640 0x7fdd3c0685e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 --2- 172.21.15.97:0/2224959898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100830 0x7fdd3c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.756+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 >> 172.21.15.97:0/2224959898 conn(0x7fdd3c0fabb0 msgr2=0x7fdd3c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.757+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 shutdown_connections 2023-10-08T10:03:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2023-10-08T10:03:08.757+0000 7fdd42a5a700 1 -- 172.21.15.97:0/2224959898 wait complete. 2023-10-08T10:03:09.575 INFO:tasks.workunit.client.0.smithi097.stdout:Syncing()...41 sec 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout:FFSB benchmark finished at: Sun Oct 8 10:03:09 2023 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout:Results: 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout:Benchmark took 347.23 sec 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout:Total Results 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout:=============== 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout: ======= ============ ========= ======= =========== ========== 2023-10-08T10:03:09.576 INFO:tasks.workunit.client.0.smithi097.stdout: write : 2561280 7376.33 100.000% 100.000% 28.8MB/sec 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout:- 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout:7376.33 Transactions per Second 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout:Throughput Results 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout:=================== 2023-10-08T10:03:09.577 INFO:tasks.workunit.client.0.smithi097.stdout:Write Throughput: 28.8MB/sec 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout:System Call Latency statistics in millisecs 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout:===== 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout: Min Avg Max Total Calls 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout: ======== ======== ======== ============ 2023-10-08T10:03:09.588 INFO:tasks.workunit.client.0.smithi097.stdout:[ open] 0.004000 1.173585 30.653999 2001 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[7] 20.000000 - 50.000000 : 9 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[8] 10.000000 - 20.000000 : 57 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[9] 5.000000 - 10.000000 : 114 2023-10-08T10:03:09.589 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[10] 2.000000 - 5.000000 : 119 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[11] 1.000000 - 2.000000 : 80 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[12] 0.500000 - 1.000000 : 71 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[14] 0.100000 - 0.200000 : 3 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[15] 0.050000 - 0.100000 : 4 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[16] 0.020000 - 0.050000 : 244 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[17] 0.010000 - 0.020000 : 608 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[18] 0.000000 - 0.010000 : 691 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout:[ write] 0.001000 2.905381 43743.359375 2561280 2023-10-08T10:03:09.590 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[0] 5000.000000 - 10000.000000 : 377 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[1] 2000.000000 - 5000.000000 : 624 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[2] 1000.000000 - 2000.000000 : 338 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[3] 500.000000 - 1000.000000 : 378 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[4] 200.000000 - 500.000000 : 474 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[5] 100.000000 - 200.000000 : 357 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[6] 50.000000 - 100.000000 : 348 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[7] 20.000000 - 50.000000 : 346 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[8] 10.000000 - 20.000000 : 177 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[9] 5.000000 - 10.000000 : 124 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[10] 2.000000 - 5.000000 : 100 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[11] 1.000000 - 2.000000 : 32 2023-10-08T10:03:09.591 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[12] 0.500000 - 1.000000 : 15 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[13] 0.200000 - 0.500000 : 59 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[14] 0.100000 - 0.200000 : 68 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[15] 0.050000 - 0.100000 : 105 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[16] 0.020000 - 0.050000 : 1662 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[17] 0.010000 - 0.020000 : 7955 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[18] 0.000000 - 0.010000 : 2547629 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout:[ lseek] 0.000000 0.884775 43947.398438 2561280 2023-10-08T10:03:09.592 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[0] 5000.000000 - 10000.000000 : 105 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[1] 2000.000000 - 5000.000000 : 154 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[2] 1000.000000 - 2000.000000 : 76 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[3] 500.000000 - 1000.000000 : 52 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[4] 200.000000 - 500.000000 : 51 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[5] 100.000000 - 200.000000 : 25 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[6] 50.000000 - 100.000000 : 56 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[7] 20.000000 - 50.000000 : 84 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[8] 10.000000 - 20.000000 : 45 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[9] 5.000000 - 10.000000 : 60 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[10] 2.000000 - 5.000000 : 41 2023-10-08T10:03:09.593 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[11] 1.000000 - 2.000000 : 15 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[12] 0.500000 - 1.000000 : 12 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[13] 0.200000 - 0.500000 : 15 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[14] 0.100000 - 0.200000 : 18 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[15] 0.050000 - 0.100000 : 20 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[16] 0.020000 - 0.050000 : 181 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[17] 0.010000 - 0.020000 : 1037 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[18] 0.000000 - 0.010000 : 2559183 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout:[ close] 0.001000 0.004047 0.135000 2001 2023-10-08T10:03:09.594 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2023-10-08T10:03:09.595 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[14] 0.100000 - 0.200000 : 1 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[16] 0.020000 - 0.050000 : 4 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[17] 0.010000 - 0.020000 : 6 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: msec_range[18] 0.000000 - 0.010000 : 1990 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.596 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.597 INFO:tasks.workunit.client.0.smithi097.stdout: 2023-10-08T10:03:09.597 INFO:tasks.workunit.client.0.smithi097.stdout:0.3% User Time 2023-10-08T10:03:09.597 INFO:tasks.workunit.client.0.smithi097.stdout:2.6% System Time 2023-10-08T10:03:09.597 INFO:tasks.workunit.client.0.smithi097.stdout:2.9% CPU Utilization 2023-10-08T10:03:09.597 INFO:tasks.workunit.client.0.smithi097.stderr:+ cd .. 2023-10-08T10:03:09.598 INFO:tasks.workunit.client.0.smithi097.stderr:+ rm -r tmp ffsb 2023-10-08T10:03:09.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:09 smithi097 ceph-mon[99491]: pgmap v540: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-10-08T10:03:09.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:09 smithi178 ceph-mon[103156]: pgmap v540: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-10-08T10:03:10.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:09 smithi186 ceph-mon[104393]: pgmap v540: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-10-08T10:03:11.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:11 smithi097 ceph-mon[99491]: pgmap v541: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-10-08T10:03:11.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:11 smithi178 ceph-mon[103156]: pgmap v541: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-10-08T10:03:11.779 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:11 smithi186 ceph-mon[104393]: pgmap v541: 129 pgs: 129 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-10-08T10:03:12.301 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-08T10:03:12.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-08T10:03:12.581 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2023-10-08T10:03:12.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-08T10:03:12.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:12 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:12.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:12 smithi097 ceph-mon[99491]: pgmap v542: 129 pgs: 129 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-10-08T10:03:13.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:12 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:13.025 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:12 smithi186 ceph-mon[104393]: pgmap v542: 129 pgs: 129 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-10-08T10:03:13.100 DEBUG:teuthology.parallel:result is None 2023-10-08T10:03:13.100 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-08T10:03:13.209 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:12 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:13.209 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:12 smithi178 ceph-mon[103156]: pgmap v542: 129 pgs: 129 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-10-08T10:03:13.235 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-08T10:03:13.235 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-08T10:03:13.247 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T10:03:13.263 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-08T10:03:13.314 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-08T10:03:13.315 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: Command failed on smithi097 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status' 2023-10-08T10:03:13.316 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-08T10:03:13.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T10:03:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.582+0000 7ff1a7725700 1 -- 172.21.15.97:0/1125959658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a00ff6f0 msgr2=0x7ff1a00ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.582+0000 7ff1a7725700 1 --2- 172.21.15.97:0/1125959658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a00ffb70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff190009990 tx=0x7ff190009ca0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.583+0000 7ff1a7725700 1 -- 172.21.15.97:0/1125959658 shutdown_connections 2023-10-08T10:03:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.583+0000 7ff1a7725700 1 --2- 172.21.15.97:0/1125959658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a01000b0 0x7ff1a0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.583+0000 7ff1a7725700 1 --2- 172.21.15.97:0/1125959658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00fe990 0x7ff1a00fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.583+0000 7ff1a7725700 1 --2- 172.21.15.97:0/1125959658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a00ffb70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.583+0000 7ff1a7725700 1 -- 172.21.15.97:0/1125959658 >> 172.21.15.97:0/1125959658 conn(0x7ff1a00fa2c0 msgr2=0x7ff1a00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.586+0000 7ff1a7725700 1 -- 172.21.15.97:0/1125959658 shutdown_connections 2023-10-08T10:03:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.589+0000 7ff1a7725700 1 -- 172.21.15.97:0/1125959658 wait complete. 2023-10-08T10:03:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.590+0000 7ff1a7725700 1 Processor -- start 2023-10-08T10:03:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.590+0000 7ff1a7725700 1 -- start start 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a01000b0 0x7ff1a019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a01a2430 0x7ff1a01a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01a4df0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01a4f30 con 0x7ff1a01000b0 2023-10-08T10:03:13.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.591+0000 7ff1a7725700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01a50c0 con 0x7ff1a01a2430 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.597+0000 7ff1a54c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.597+0000 7ff1a54c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58620/0 (socket says 172.21.15.97:58620) 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.597+0000 7ff1a54c1700 1 -- 172.21.15.97:0/3924783191 learned_addr learned my addr 172.21.15.97:0/3924783191 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 -- 172.21.15.97:0/3924783191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a01a2430 msgr2=0x7ff1a01a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a01a2430 0x7ff1a01a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 -- 172.21.15.97:0/3924783191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a01000b0 msgr2=0x7ff1a019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:13.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a01000b0 0x7ff1a019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:13.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 -- 172.21.15.97:0/3924783191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900095f0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.598+0000 7ff1a54c1700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00b450 tx=0x7ff19c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:13.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.599+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c011630 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.599+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c011c50 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.599+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c010e00 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.601+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a01a53a0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.601+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a58b0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.604+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a01040e0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.618+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff19c011790 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.619+0000 7ff1967fc700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 0x7ff18c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:13.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.619+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff19c0ccf10 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.619+0000 7ff1a4cc0700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 0x7ff18c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:13.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.619+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff19c0cd3f0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.626+0000 7ff1a4cc0700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 0x7ff18c0a3a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098e0 tx=0x7ff190005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.993+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a00678a0 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:13.996+0000 7ff1967fc700 1 -- 172.21.15.97:0/3924783191 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7ff19c015070 con 0x7ff1a00ff6f0 2023-10-08T10:03:13.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:13.998 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T10:03:14.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.006+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 msgr2=0x7ff18c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.006+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 0x7ff18c0a3a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098e0 tx=0x7ff190005fd0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.006+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 msgr2=0x7ff1a019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.006+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00b450 tx=0x7ff19c00b810 comp rx=0 tx=0).stop 2023-10-08T10:03:14.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 shutdown_connections 2023-10-08T10:03:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff18c0a15d0 0x7ff18c0a3a90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1a01a2430 0x7ff1a01a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a00ff6f0 0x7ff1a019cee0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 --2- 172.21.15.97:0/3924783191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1a01000b0 0x7ff1a019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.014+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 >> 172.21.15.97:0/3924783191 conn(0x7ff1a00fa2c0 msgr2=0x7ff1a00faf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:14.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.026+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 shutdown_connections 2023-10-08T10:03:14.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.030+0000 7ff1a7725700 1 -- 172.21.15.97:0/3924783191 wait complete. 2023-10-08T10:03:14.073 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T10:03:14.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T10:03:14.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.358+0000 7f2c48c50700 1 -- 172.21.15.97:0/2948495503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c441088a0 msgr2=0x7f2c44108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.358+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2948495503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c441088a0 0x7f2c44108d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800aaa0 tx=0x7f2c3800adb0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.371+0000 7f2c48c50700 1 -- 172.21.15.97:0/2948495503 shutdown_connections 2023-10-08T10:03:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.371+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2948495503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c4410faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.371+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2948495503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c44107b40 0x7f2c44107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.371+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2948495503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c441088a0 0x7f2c44108d20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.371+0000 7f2c48c50700 1 -- 172.21.15.97:0/2948495503 >> 172.21.15.97:0/2948495503 conn(0x7f2c4406d790 msgr2=0x7f2c4406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:14.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.375+0000 7f2c48c50700 1 -- 172.21.15.97:0/2948495503 shutdown_connections 2023-10-08T10:03:14.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.378+0000 7f2c48c50700 1 -- 172.21.15.97:0/2948495503 wait complete. 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 Processor -- start 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 -- start start 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 0x7f2c441a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 0x7f2c441a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c441acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:14.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c441ad1e0 con 0x7f2c441088a0 2023-10-08T10:03:14.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c441ad320 con 0x7f2c44107b40 2023-10-08T10:03:14.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.379+0000 7f2c48c50700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c441ad4e0 con 0x7f2c44109260 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.380+0000 7f2c42d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c441acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.380+0000 7f2c42d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c441acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.97:60332/0 (socket says 172.21.15.97:60332) 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.380+0000 7f2c42d9d700 1 -- 172.21.15.97:0/2578787981 learned_addr learned my addr 172.21.15.97:0/2578787981 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.380+0000 7f2c4259c700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 0x7f2c441a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 0x7f2c441a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 -- 172.21.15.97:0/2578787981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 msgr2=0x7f2c441acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c441acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 -- 172.21.15.97:0/2578787981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 msgr2=0x7f2c441a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 0x7f2c441a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 -- 172.21.15.97:0/2578787981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3800a700 con 0x7f2c441088a0 2023-10-08T10:03:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c4259c700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 0x7f2c441a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.381+0000 7f2c41d9b700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 0x7f2c441a5c40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2c38006010 tx=0x7f2c38003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.382+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380040d0 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.382+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c38004230 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.382+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c44117b10 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.382+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38016680 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.382+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c44118010 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.383+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c44067530 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.384+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2c380167e0 con 0x7f2c441088a0 2023-10-08T10:03:14.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.385+0000 7f2c337fe700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 0x7f2c2c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:14.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.385+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2c38012070 con 0x7f2c441088a0 2023-10-08T10:03:14.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.389+0000 7f2c4259c700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 0x7f2c2c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:14.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.391+0000 7f2c4259c700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 0x7f2c2c0b49d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c34006fd0 tx=0x7f2c34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:14.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.391+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2c380982c0 con 0x7f2c441088a0 2023-10-08T10:03:14.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:14 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3924783191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:14.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:14 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3924783191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:14.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:14 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3924783191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:14.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.752+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c44118460 con 0x7f2c441088a0 2023-10-08T10:03:14.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.753+0000 7f2c337fe700 1 -- 172.21.15.97:0/2578787981 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c3809b8e0 con 0x7f2c441088a0 2023-10-08T10:03:14.754 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T10:03:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.769+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 msgr2=0x7f2c2c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.769+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 0x7f2c2c0b49d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c34006fd0 tx=0x7f2c34008040 comp rx=0 tx=0).stop 2023-10-08T10:03:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.769+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 msgr2=0x7f2c441a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.769+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 0x7f2c441a5c40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2c38006010 tx=0x7f2c38003910 comp rx=0 tx=0).stop 2023-10-08T10:03:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 shutdown_connections 2023-10-08T10:03:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2c2c0b2510 0x7f2c2c0b49d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c44109260 0x7f2c441acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c44107b40 0x7f2c441a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 --2- 172.21.15.97:0/2578787981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c441088a0 0x7f2c441a5c40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:14.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.774+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 >> 172.21.15.97:0/2578787981 conn(0x7f2c4406d790 msgr2=0x7f2c44070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.784+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 shutdown_connections 2023-10-08T10:03:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:14.784+0000 7f2c48c50700 1 -- 172.21.15.97:0/2578787981 wait complete. 2023-10-08T10:03:14.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T10:03:15.091 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:15 smithi097 ceph-mon[99491]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 140 KiB/s wr, 66 op/s 2023-10-08T10:03:15.092 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:15 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2578787981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:15.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.137+0000 7fed9a3bc700 1 -- 172.21.15.97:0/1089898970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed9410a160 msgr2=0x7fed9410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.137+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/1089898970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed9410a160 0x7fed9410c5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed880098f0 tx=0x7fed88009c00 comp rx=0 tx=0).stop 2023-10-08T10:03:15.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 -- 172.21.15.97:0/1089898970 shutdown_connections 2023-10-08T10:03:15.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/1089898970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/1089898970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941077f0 0x7fed94109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/1089898970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed9410a160 0x7fed9410c5f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 -- 172.21.15.97:0/1089898970 >> 172.21.15.97:0/1089898970 conn(0x7fed9406d790 msgr2=0x7fed9406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:15.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 -- 172.21.15.97:0/1089898970 shutdown_connections 2023-10-08T10:03:15.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.138+0000 7fed9a3bc700 1 -- 172.21.15.97:0/1089898970 wait complete. 2023-10-08T10:03:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.143+0000 7fed9a3bc700 1 Processor -- start 2023-10-08T10:03:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.143+0000 7fed9a3bc700 1 -- start start 2023-10-08T10:03:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed941077f0 0x7fed9419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 0x7fed941a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed941a4f70 con 0x7fed941a25b0 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed941a50b0 con 0x7fed941077f0 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.144+0000 7fed9a3bc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed941a5240 con 0x7fed9410cb30 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.145+0000 7fed937fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.145+0000 7fed98959700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 0x7fed941a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.145+0000 7fed98959700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 0x7fed941a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58644/0 (socket says 172.21.15.97:58644) 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.145+0000 7fed98959700 1 -- 172.21.15.97:0/3883531649 learned_addr learned my addr 172.21.15.97:0/3883531649 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 -- 172.21.15.97:0/3883531649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed941077f0 msgr2=0x7fed9419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed941077f0 0x7fed9419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 -- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 msgr2=0x7fed941a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 0x7fed941a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 -- 172.21.15.97:0/3883531649 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed88009520 con 0x7fed9410cb30 2023-10-08T10:03:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.146+0000 7fed937fe700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9419d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed88008000 tx=0x7fed8800fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.149+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800bac0 con 0x7fed9410cb30 2023-10-08T10:03:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.150+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed8803e070 con 0x7fed9410cb30 2023-10-08T10:03:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.150+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88003d30 con 0x7fed9410cb30 2023-10-08T10:03:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.153+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94072a20 con 0x7fed9410cb30 2023-10-08T10:03:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.153+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94072e50 con 0x7fed9410cb30 2023-10-08T10:03:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.154+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed94067530 con 0x7fed9410cb30 2023-10-08T10:03:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.164+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fed88040070 con 0x7fed9410cb30 2023-10-08T10:03:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.164+0000 7fed917fa700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 0x7fed7c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.165+0000 7fed93fff700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 0x7fed7c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.165+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed880f0080 con 0x7fed9410cb30 2023-10-08T10:03:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.171+0000 7fed93fff700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 0x7fed7c0a3b40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00ab90 tx=0x7fed8c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.177+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fed880b9460 con 0x7fed9410cb30 2023-10-08T10:03:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:15 smithi178 ceph-mon[103156]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 140 KiB/s wr, 66 op/s 2023-10-08T10:03:15.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:15 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2578787981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:15 smithi186 ceph-mon[104393]: pgmap v543: 129 pgs: 129 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 140 KiB/s wr, 66 op/s 2023-10-08T10:03:15.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:15 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2578787981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:15.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.544+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed94110790 con 0x7fed9410cb30 2023-10-08T10:03:15.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.566+0000 7fed917fa700 1 -- 172.21.15.97:0/3883531649 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed880bca80 con 0x7fed9410cb30 2023-10-08T10:03:15.571 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T10:03:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.577+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 msgr2=0x7fed7c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.577+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 0x7fed7c0a3b40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00ab90 tx=0x7fed8c009250 comp rx=0 tx=0).stop 2023-10-08T10:03:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.577+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 msgr2=0x7fed9419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.577+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9419d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed88008000 tx=0x7fed8800fa40 comp rx=0 tx=0).stop 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 shutdown_connections 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fed7c0a1680 0x7fed7c0a3b40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed941a25b0 0x7fed941a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed941077f0 0x7fed9419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 --2- 172.21.15.97:0/3883531649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed9410cb30 0x7fed9419d5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.581+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 >> 172.21.15.97:0/3883531649 conn(0x7fed9406d790 msgr2=0x7fed9410aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.586+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 shutdown_connections 2023-10-08T10:03:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.586+0000 7fed9a3bc700 1 -- 172.21.15.97:0/3883531649 wait complete. 2023-10-08T10:03:15.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 -- 172.21.15.97:0/2795003746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc10a160 msgr2=0x7fe4dc10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/2795003746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc10a160 0x7fe4dc10c5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009f90 tx=0x7fe4d400b330 comp rx=0 tx=0).stop 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 -- 172.21.15.97:0/2795003746 shutdown_connections 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/2795003746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/2795003746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc1077f0 0x7fe4dc109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/2795003746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc10a160 0x7fe4dc10c5f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.873+0000 7fe4e21cc700 1 -- 172.21.15.97:0/2795003746 >> 172.21.15.97:0/2795003746 conn(0x7fe4dc06d790 msgr2=0x7fe4dc06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:15.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.878+0000 7fe4e21cc700 1 -- 172.21.15.97:0/2795003746 shutdown_connections 2023-10-08T10:03:15.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.879+0000 7fe4e21cc700 1 -- 172.21.15.97:0/2795003746 wait complete. 2023-10-08T10:03:15.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.880+0000 7fe4e21cc700 1 Processor -- start 2023-10-08T10:03:15.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.882+0000 7fe4e21cc700 1 -- start start 2023-10-08T10:03:15.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc1077f0 0x7fe4dc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc10a160 0x7fe4dc19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a4890 con 0x7fe4dc10a160 2023-10-08T10:03:15.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a4a00 con 0x7fe4dc10cb30 2023-10-08T10:03:15.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.883+0000 7fe4e21cc700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a4bc0 con 0x7fe4dc1077f0 2023-10-08T10:03:15.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.884+0000 7fe4dbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:15.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.884+0000 7fe4dbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38626/0 (socket says 172.21.15.97:38626) 2023-10-08T10:03:15.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.884+0000 7fe4dbfff700 1 -- 172.21.15.97:0/3122828422 learned_addr learned my addr 172.21.15.97:0/3122828422 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:15.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.888+0000 7fe4dbfff700 1 -- 172.21.15.97:0/3122828422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc1077f0 msgr2=0x7fe4dc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:15.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.888+0000 7fe4dbfff700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc1077f0 0x7fe4dc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.888+0000 7fe4dbfff700 1 -- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc10a160 msgr2=0x7fe4dc19d2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T10:03:15.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.888+0000 7fe4dbfff700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc10a160 0x7fe4dc19d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:15.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.888+0000 7fe4dbfff700 1 -- 172.21.15.97:0/3122828422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009c40 con 0x7fe4dc10cb30 2023-10-08T10:03:15.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.890+0000 7fe4dbfff700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01fac0 tx=0x7fe4cc01fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.894+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc01da40 con 0x7fe4dc10cb30 2023-10-08T10:03:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.894+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4cc004cf0 con 0x7fe4dc10cb30 2023-10-08T10:03:15.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.894+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc005650 con 0x7fe4dc10cb30 2023-10-08T10:03:15.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.901+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc1a4ea0 con 0x7fe4dc10cb30 2023-10-08T10:03:15.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.901+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1a52d0 con 0x7fe4dc10cb30 2023-10-08T10:03:15.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.902+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc067530 con 0x7fe4dc10cb30 2023-10-08T10:03:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.906+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe4cc01dca0 con 0x7fe4dc10cb30 2023-10-08T10:03:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.907+0000 7fe4d8ff9700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 0x7fe4c40a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.907+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe4cc025070 con 0x7fe4dc10cb30 2023-10-08T10:03:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.910+0000 7fe4db7fe700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 0x7fe4c40a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:15.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.911+0000 7fe4db7fe700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 0x7fe4c40a3ac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc19e220 tx=0x7fe4d0006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:15.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:15.914+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4cc0a7950 con 0x7fe4dc10cb30 2023-10-08T10:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.283+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4dc110800 con 0x7fe4dc10cb30 2023-10-08T10:03:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.284+0000 7fe4d8ff9700 1 -- 172.21.15.97:0/3122828422 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4cc0aaf70 con 0x7fe4dc10cb30 2023-10-08T10:03:16.285 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T10:03:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.287+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 msgr2=0x7fe4c40a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.287+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 0x7fe4c40a3ac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc19e220 tx=0x7fe4d0006cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.287+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 msgr2=0x7fe4dc1a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.287+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01fac0 tx=0x7fe4cc01fe80 comp rx=0 tx=0).stop 2023-10-08T10:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 shutdown_connections 2023-10-08T10:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4c40a1600 0x7fe4c40a3ac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc10cb30 0x7fe4dc1a42c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4dc1077f0 0x7fe4dc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 --2- 172.21.15.97:0/3122828422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4dc10a160 0x7fe4dc19d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 >> 172.21.15.97:0/3122828422 conn(0x7fe4dc06d790 msgr2=0x7fe4dc10a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:16.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.289+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 shutdown_connections 2023-10-08T10:03:16.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.292+0000 7fe4e21cc700 1 -- 172.21.15.97:0/3122828422 wait complete. 2023-10-08T10:03:16.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T10:03:16.402 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:16 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:16.402 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:16 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3883531649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:16.459 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:16 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:16.459 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:16 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3883531649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:16 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:16.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:16 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3883531649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 -- 172.21.15.97:0/113102673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190106dc0 msgr2=0x7f71901071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 --2- 172.21.15.97:0/113102673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190106dc0 0x7f71901071e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f718800b0a0 tx=0x7f718800b3b0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 -- 172.21.15.97:0/113102673 shutdown_connections 2023-10-08T10:03:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 --2- 172.21.15.97:0/113102673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f719010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 --2- 172.21.15.97:0/113102673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190106dc0 0x7f71901071e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 --2- 172.21.15.97:0/113102673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190107fb0 0x7f7190108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.605+0000 7f719586a700 1 -- 172.21.15.97:0/113102673 >> 172.21.15.97:0/113102673 conn(0x7f7190075d60 msgr2=0x7f71900781a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.611+0000 7f719586a700 1 -- 172.21.15.97:0/113102673 shutdown_connections 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.611+0000 7f719586a700 1 -- 172.21.15.97:0/113102673 wait complete. 2023-10-08T10:03:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 Processor -- start 2023-10-08T10:03:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 -- start start 2023-10-08T10:03:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190106dc0 0x7f719019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 0x7f719019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f71901a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901a4a00 con 0x7f7190106dc0 2023-10-08T10:03:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901a4b40 con 0x7f7190108950 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f719586a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901a4cd0 con 0x7f7190107fb0 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f71901a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f71901a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.97:38634/0 (socket says 172.21.15.97:38634) 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.612+0000 7f718f7fe700 1 -- 172.21.15.97:0/2495534585 learned_addr learned my addr 172.21.15.97:0/2495534585 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.617+0000 7f718effd700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190106dc0 0x7f719019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.618+0000 7f718e7fc700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 0x7f719019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 -- 172.21.15.97:0/2495534585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 msgr2=0x7f71901a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f71901a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 -- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190106dc0 msgr2=0x7f719019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190106dc0 0x7f719019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 -- 172.21.15.97:0/2495534585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188009d00 con 0x7f7190107fb0 2023-10-08T10:03:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.619+0000 7f718e7fc700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 0x7f719019d410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7184006fd0 tx=0x7f718400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.620+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184011830 con 0x7f7190107fb0 2023-10-08T10:03:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.620+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7184011e50 con 0x7f7190107fb0 2023-10-08T10:03:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.620+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184010d70 con 0x7f7190107fb0 2023-10-08T10:03:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.622+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71901a4fb0 con 0x7f7190107fb0 2023-10-08T10:03:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.622+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71901a5320 con 0x7f7190107fb0 2023-10-08T10:03:16.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.625+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71740052f0 con 0x7f7190107fb0 2023-10-08T10:03:16.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.633+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7184011990 con 0x7f7190107fb0 2023-10-08T10:03:16.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.633+0000 7f7194868700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 0x7f71780a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.634+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f71840cccb0 con 0x7f7190107fb0 2023-10-08T10:03:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.634+0000 7f718effd700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 0x7f71780a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.634+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f71840cd190 con 0x7f7190107fb0 2023-10-08T10:03:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:16.634+0000 7f718effd700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 0x7f71780a3ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7188007c00 tx=0x7f7188007cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.013+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7174005b20 con 0x7f7190107fb0 2023-10-08T10:03:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.019+0000 7f7194868700 1 -- 172.21.15.97:0/2495534585 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7184015020 con 0x7f7190107fb0 2023-10-08T10:03:17.027 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T10:03:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.030+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 msgr2=0x7f71780a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.030+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 0x7f71780a3ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7188007c00 tx=0x7f7188007cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.030+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 msgr2=0x7f719019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.030+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 0x7f719019d410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7184006fd0 tx=0x7f718400b730 comp rx=0 tx=0).stop 2023-10-08T10:03:17.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 shutdown_connections 2023-10-08T10:03:17.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f71780a1600 0x7f71780a3ac0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7190108950 0x7f71901a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190106dc0 0x7f719019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 --2- 172.21.15.97:0/2495534585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7190107fb0 0x7f719019d410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.043+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 >> 172.21.15.97:0/2495534585 conn(0x7f7190075d60 msgr2=0x7f719010b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:17.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.045+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 shutdown_connections 2023-10-08T10:03:17.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.045+0000 7f719586a700 1 -- 172.21.15.97:0/2495534585 wait complete. 2023-10-08T10:03:17.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-08T10:03:17.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.323+0000 7fe490538700 1 -- 172.21.15.97:0/4315191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 msgr2=0x7fe48807a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.323+0000 7fe490538700 1 --2- 172.21.15.97:0/4315191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe48807a280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe478009ca0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.324+0000 7fe490538700 1 -- 172.21.15.97:0/4315191 shutdown_connections 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.324+0000 7fe490538700 1 --2- 172.21.15.97:0/4315191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe48807a280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.324+0000 7fe490538700 1 --2- 172.21.15.97:0/4315191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488072540 0x7fe488072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.324+0000 7fe490538700 1 --2- 172.21.15.97:0/4315191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488075420 0x7fe4880778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.324+0000 7fe490538700 1 -- 172.21.15.97:0/4315191 >> 172.21.15.97:0/4315191 conn(0x7fe48806d6c0 msgr2=0x7fe48806fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.332+0000 7fe490538700 1 -- 172.21.15.97:0/4315191 shutdown_connections 2023-10-08T10:03:17.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.332+0000 7fe490538700 1 -- 172.21.15.97:0/4315191 wait complete. 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.333+0000 7fe490538700 1 Processor -- start 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 -- start start 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488072540 0x7fe48812b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 0x7fe48812dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe4881302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488130a10 con 0x7fe488075420 2023-10-08T10:03:17.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488130b80 con 0x7fe488077df0 2023-10-08T10:03:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe490538700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488130d40 con 0x7fe488072540 2023-10-08T10:03:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe48dad3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 0x7fe48812dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe48dad3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 0x7fe48812dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58678/0 (socket says 172.21.15.97:58678) 2023-10-08T10:03:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.334+0000 7fe48dad3700 1 -- 172.21.15.97:0/3666642790 learned_addr learned my addr 172.21.15.97:0/3666642790 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe4881302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48e2d4700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488072540 0x7fe48812b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 -- 172.21.15.97:0/3666642790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488072540 msgr2=0x7fe48812b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:17.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488072540 0x7fe48812b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 -- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 msgr2=0x7fe48812dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:17.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 0x7fe48812dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:17.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.335+0000 7fe48ead5700 1 -- 172.21.15.97:0/3666642790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009c40 con 0x7fe488077df0 2023-10-08T10:03:17.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.337+0000 7fe48ead5700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe4881302b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe47800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:17.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.341+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47801d070 con 0x7fe488077df0 2023-10-08T10:03:17.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.341+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe478003d50 con 0x7fe488077df0 2023-10-08T10:03:17.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.341+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478022e60 con 0x7fe488077df0 2023-10-08T10:03:17.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.341+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4780095f0 con 0x7fe488077df0 2023-10-08T10:03:17.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.341+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488131280 con 0x7fe488077df0 2023-10-08T10:03:17.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.344+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe478003890 con 0x7fe488077df0 2023-10-08T10:03:17.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.345+0000 7fe47f7fe700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 0x7fe4740a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:17.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.345+0000 7fe48e2d4700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 0x7fe4740a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:17.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.345+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe4780ce280 con 0x7fe488077df0 2023-10-08T10:03:17.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.346+0000 7fe48e2d4700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 0x7fe4740a39b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe484009a10 tx=0x7fe484009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:17.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.348+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488067450 con 0x7fe488077df0 2023-10-08T10:03:17.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.362+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe47805d100 con 0x7fe488077df0 2023-10-08T10:03:17.454 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:17 smithi097 ceph-mon[99491]: pgmap v544: 129 pgs: 129 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 143 KiB/s wr, 76 op/s 2023-10-08T10:03:17.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3122828422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:17 smithi097 ceph-mon[99491]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:17.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:17 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2495534585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:17 smithi178 ceph-mon[103156]: pgmap v544: 129 pgs: 129 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 143 KiB/s wr, 76 op/s 2023-10-08T10:03:17.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3122828422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:17 smithi178 ceph-mon[103156]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:17.458 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:17 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2495534585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:17 smithi186 ceph-mon[104393]: pgmap v544: 129 pgs: 129 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 143 KiB/s wr, 76 op/s 2023-10-08T10:03:17.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3122828422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:17 smithi186 ceph-mon[104393]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T10:03:17.526 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:17 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2495534585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:17.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.809+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fe48807d880 con 0x7fe488077df0 2023-10-08T10:03:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.826+0000 7fe47f7fe700 1 -- 172.21.15.97:0/3666642790 <== mon.2 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v16) v1 ==== 81+0+733878 (secure 0 0 0) 0x7fe478027020 con 0x7fe488077df0 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 16, 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "active_gid": 14152, 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "active_name": "x", 2023-10-08T10:03:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addrs": { 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6800", 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3991091613 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6801", 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3991091613 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.836 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addr": "172.21.15.97:6801/3991091613", 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "active_change": "2023-10-08T09:47:45.797769+0000", 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "active_mgr_features": 4540138322906710015, 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "available": true, 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "standbys": [ 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24109, 2023-10-08T10:03:17.837 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "z", 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540138322906710015, 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.838 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.839 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.840 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.841 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.844 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.845 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2023-10-08T10:03:17.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.848 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2023-10-08T10:03:17.849 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.850 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.851 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2023-10-08T10:03:17.852 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.854 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.855 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.857 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.858 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2023-10-08T10:03:17.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.860 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T10:03:17.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2023-10-08T10:03:17.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2023-10-08T10:03:17.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.865 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.866 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.867 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.869 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.870 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.871 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.872 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.874 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap" 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.875 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T10:03:17.876 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.877 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:17.878 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2023-10-08T10:03:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.883 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.885 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.886 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2023-10-08T10:03:17.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T10:03:17.888 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.889 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2023-10-08T10:03:17.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.891 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T10:03:17.893 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.894 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.896 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.897 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.899 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.900 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.901 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2023-10-08T10:03:17.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.904 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.905 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.906 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.907 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.908 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2023-10-08T10:03:17.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.913 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2023-10-08T10:03:17.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2023-10-08T10:03:17.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2023-10-08T10:03:17.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.920 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.921 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.922 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.923 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.924 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.925 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.926 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.927 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.928 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.930 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.931 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2023-10-08T10:03:17.932 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.933 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.934 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:17.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.937 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2023-10-08T10:03:17.940 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.943 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2023-10-08T10:03:17.944 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.945 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.946 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2023-10-08T10:03:17.947 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.949 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.950 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.951 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.955 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.956 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.959 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2023-10-08T10:03:17.961 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.962 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.963 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.964 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.965 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T10:03:17.966 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.967 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.968 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.969 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.970 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.972 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.973 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.975 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T10:03:17.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.977 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.978 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.979 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.980 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.981 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.983 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.986 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T10:03:17.988 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.990 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T10:03:17.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.995 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.996 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T10:03:17.998 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:17.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.001 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.002 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.003 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.004 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.006 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.007 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:18.008 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T10:03:18.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.011 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.012 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.013 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.015 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2023-10-08T10:03:18.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2023-10-08T10:03:18.019 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.021 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.024 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.025 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.026 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.027 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2023-10-08T10:03:18.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.029 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.030 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.031 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2023-10-08T10:03:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2023-10-08T10:03:18.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.035 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.036 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:18.037 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.038 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2023-10-08T10:03:18.039 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.043 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.044 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.045 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.046 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.048 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.049 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.052 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.053 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.054 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.057 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.059 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.062 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.063 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.064 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.065 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.066 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.067 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.068 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.071 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.073 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.074 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2023-10-08T10:03:18.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2023-10-08T10:03:18.078 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.080 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2023-10-08T10:03:18.082 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.084 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.086 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.087 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.088 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.089 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2023-10-08T10:03:18.090 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.091 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.094 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.095 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.096 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.097 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.099 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.101 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.104 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.105 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2023-10-08T10:03:18.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.108 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.109 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.111 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.112 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2023-10-08T10:03:18.115 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2023-10-08T10:03:18.117 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.118 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.119 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T10:03:18.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.126 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.127 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.130 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.131 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.132 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2023-10-08T10:03:18.133 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.134 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.135 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.136 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.137 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.141 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.142 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.143 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.144 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.145 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.149 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.152 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.153 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:18.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.155 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.156 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.157 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.159 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.160 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.161 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.162 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.163 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.164 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.167 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.170 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.172 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.173 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.174 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.175 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.177 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.180 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.182 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.183 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.186 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.187 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.188 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.190 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.192 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2023-10-08T10:03:18.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.197 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.198 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.201 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2023-10-08T10:03:18.202 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.204 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.205 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.206 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2023-10-08T10:03:18.207 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.209 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.210 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.212 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.213 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.215 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.216 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.217 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T10:03:18.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.221 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.222 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.226 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.227 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.229 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.232 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.233 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.234 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.235 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.238 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "drive_group_interval": { 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "drive_group_interval", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300.0", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.241 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.242 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.243 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.244 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.245 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.247 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2023-10-08T10:03:18.248 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2023-10-08T10:03:18.249 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.250 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.251 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.252 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.254 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.256 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2023-10-08T10:03:18.257 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2023-10-08T10:03:18.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.260 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.261 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.262 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.263 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.264 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2023-10-08T10:03:18.265 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.280 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.282 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.283 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.287 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.288 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.293 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.295 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.296 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.297 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.299 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2023-10-08T10:03:18.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.304 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T10:03:18.305 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.306 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.307 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.308 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2023-10-08T10:03:18.309 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.312 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.316 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.317 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.318 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.319 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.322 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.323 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.324 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2023-10-08T10:03:18.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.326 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.327 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.329 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.330 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.332 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.333 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.334 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.336 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.338 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.339 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2023-10-08T10:03:18.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.343 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T10:03:18.344 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2023-10-08T10:03:18.345 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.349 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.350 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.351 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.354 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.355 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2023-10-08T10:03:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24119, 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "y", 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540138322906710015, 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.359 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T10:03:18.360 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.361 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.363 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.364 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.365 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.366 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.367 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.368 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.369 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.370 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2023-10-08T10:03:18.371 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.372 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.373 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.374 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.375 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2023-10-08T10:03:18.376 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T10:03:18.377 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.379 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2023-10-08T10:03:18.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2023-10-08T10:03:18.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.385 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.387 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.389 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.390 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.391 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.393 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.394 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2023-10-08T10:03:18.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.397 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap" 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2023-10-08T10:03:18.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.399 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T10:03:18.400 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.401 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.402 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.403 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.404 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.405 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.407 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T10:03:18.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.409 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.410 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.412 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T10:03:18.413 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.414 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2023-10-08T10:03:18.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.417 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.419 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.420 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.422 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.423 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.425 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.426 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2023-10-08T10:03:18.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.428 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.429 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.430 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2023-10-08T10:03:18.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.433 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.434 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.435 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T10:03:18.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.440 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.441 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.442 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.443 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.445 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.446 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.447 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.449 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.452 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.453 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.455 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.457 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:18.459 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T10:03:18.460 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.461 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.462 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.463 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2023-10-08T10:03:18.464 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.467 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.468 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.469 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2023-10-08T10:03:18.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.472 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.473 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2023-10-08T10:03:18.474 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T10:03:18.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.477 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.478 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.480 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.481 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2023-10-08T10:03:18.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2023-10-08T10:03:18.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.485 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T10:03:18.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.489 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.492 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T10:03:18.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T10:03:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.504 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T10:03:18.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.512 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T10:03:18.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.516 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T10:03:18.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T10:03:18.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:18.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.534 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2023-10-08T10:03:18.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:18.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.546 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.549 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2023-10-08T10:03:18.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:18.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.558 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2023-10-08T10:03:18.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.565 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.566 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.568 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2023-10-08T10:03:18.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2023-10-08T10:03:18.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.586 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2023-10-08T10:03:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2023-10-08T10:03:18.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T10:03:18.592 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.597 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.602 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.603 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2023-10-08T10:03:18.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.605 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2023-10-08T10:03:18.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2023-10-08T10:03:18.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.608 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.610 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.617 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.620 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.624 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2023-10-08T10:03:18.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.630 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.633 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.636 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.637 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.638 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.641 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.643 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2023-10-08T10:03:18.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.648 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.651 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.652 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.653 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.655 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.658 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.659 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.662 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.666 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.667 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.670 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.671 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.673 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.674 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.675 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.676 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.678 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.679 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.681 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.682 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.683 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.684 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.685 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.686 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.688 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.689 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.690 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.691 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.692 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.693 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.694 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.695 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.697 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.698 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.699 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.700 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.701 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2023-10-08T10:03:18.703 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:18.706 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.708 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.710 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.711 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.712 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2023-10-08T10:03:18.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2023-10-08T10:03:18.715 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2023-10-08T10:03:18.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T10:03:18.718 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.719 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.721 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.723 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.724 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.725 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.727 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.731 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.732 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:18.733 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.735 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.736 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.737 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.738 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.740 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.741 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T10:03:18.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.744 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.745 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.746 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.747 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.748 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.751 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.753 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.754 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.756 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.757 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.760 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.761 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.762 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "drive_group_interval": { 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "drive_group_interval", 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.763 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300.0", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.765 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.766 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.767 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.768 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.770 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.771 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.772 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.773 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.775 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.776 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.778 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.779 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.780 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.781 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2023-10-08T10:03:18.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.783 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.784 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.785 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.786 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2023-10-08T10:03:18.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.789 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.791 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2023-10-08T10:03:18.793 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.794 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.795 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.798 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.800 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.801 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2023-10-08T10:03:18.802 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.803 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.804 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.805 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.806 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.807 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.809 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.810 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.811 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.813 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.814 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.816 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.817 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.818 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.821 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.822 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.823 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2023-10-08T10:03:18.825 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.827 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.828 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.829 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2023-10-08T10:03:18.830 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.832 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.833 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.834 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.835 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.837 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.838 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.839 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.841 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.842 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.845 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.846 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.846 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.846 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2023-10-08T10:03:18.850 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.851 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.854 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.855 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.857 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.858 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.859 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.863 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.864 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.866 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.867 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.868 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.869 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.870 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2023-10-08T10:03:18.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.873 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.874 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.875 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.876 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.879 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.881 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.882 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.883 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "modules": [ 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard", 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "iostat", 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "nfs", 2023-10-08T10:03:18.885 INFO:teuthology.orchestra.run.smithi097.stdout: "restful" 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.890 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.891 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.892 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.893 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2023-10-08T10:03:18.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.896 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.897 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.898 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.899 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.900 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.901 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.903 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.904 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2023-10-08T10:03:18.908 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.909 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.910 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.911 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.912 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2023-10-08T10:03:18.919 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap" 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.921 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:18.923 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.926 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2023-10-08T10:03:18.927 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.928 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.930 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.933 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.934 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2023-10-08T10:03:18.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2023-10-08T10:03:18.936 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.937 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.938 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.939 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.940 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.942 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.943 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2023-10-08T10:03:18.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2023-10-08T10:03:18.945 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.948 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.949 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.951 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2023-10-08T10:03:18.952 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.953 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.954 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.956 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.957 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2023-10-08T10:03:18.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.961 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.962 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.963 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.964 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.965 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.967 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.968 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.970 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.971 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T10:03:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2023-10-08T10:03:18.974 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.976 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2023-10-08T10:03:18.977 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.978 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.980 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2023-10-08T10:03:18.981 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.984 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2023-10-08T10:03:18.986 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.988 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2023-10-08T10:03:18.989 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.992 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:18.993 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2023-10-08T10:03:18.994 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.995 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.997 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:18.998 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:18.999 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.000 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2023-10-08T10:03:19.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.002 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.004 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T10:03:19.006 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.008 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.010 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.011 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.013 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.013 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.013 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.013 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.014 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.015 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T10:03:19.016 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.019 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.020 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T10:03:19.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2023-10-08T10:03:19.023 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.026 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.027 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.028 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T10:03:19.030 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.031 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.033 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.035 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.038 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.039 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.040 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.041 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.042 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.043 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T10:03:19.045 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.047 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.049 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.050 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.051 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:19.052 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T10:03:19.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2023-10-08T10:03:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.058 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.059 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.060 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.061 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.062 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.063 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.065 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.067 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:19.069 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.071 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2023-10-08T10:03:19.073 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.074 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.075 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.076 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.077 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2023-10-08T10:03:19.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.079 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.080 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.081 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.082 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.083 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.085 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.086 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2023-10-08T10:03:19.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2023-10-08T10:03:19.089 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.092 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.093 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:19.094 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2023-10-08T10:03:19.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.097 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.098 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.099 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.100 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.101 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.102 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.104 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.106 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2023-10-08T10:03:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T10:03:19.109 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.110 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2023-10-08T10:03:19.111 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.112 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.113 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.115 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.116 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.117 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.118 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.119 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.121 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.122 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.123 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.128 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.130 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.131 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.132 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.133 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.136 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.137 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.138 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2023-10-08T10:03:19.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.141 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.142 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.143 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.144 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.145 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.149 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.150 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.152 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.153 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.154 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.155 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.156 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.157 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 msgr2=0x7fe4740a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 0x7fe4740a39b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe484009a10 tx=0x7fe484009190 comp rx=0 tx=0).stop 2023-10-08T10:03:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 msgr2=0x7fe4881302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe4881302b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe47800b9e0 comp rx=0 tx=0).stop 2023-10-08T10:03:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 shutdown_connections 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe4740a14f0 0x7fe4740a39b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe488077df0 0x7fe4881302b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe488072540 0x7fe48812b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 --2- 172.21.15.97:0/3666642790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe488075420 0x7fe48812dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.857+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 >> 172.21.15.97:0/3666642790 conn(0x7fe48806d6c0 msgr2=0x7fe488075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.872+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 shutdown_connections 2023-10-08T10:03:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:17.878+0000 7fe490538700 1 -- 172.21.15.97:0/3666642790 wait complete. 2023-10-08T10:03:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.162 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.163 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2023-10-08T10:03:19.164 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.165 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.166 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.168 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.171 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.172 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.173 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.174 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.175 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.176 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.180 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.181 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.182 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.184 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.185 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.188 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.189 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2023-10-08T10:03:19.191 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.192 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.195 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.196 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.197 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2023-10-08T10:03:19.198 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.199 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.201 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.202 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.204 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.205 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.206 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.207 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.208 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.209 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.210 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.212 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.213 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T10:03:19.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.216 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.216 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.216 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.217 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.218 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.219 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.220 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.221 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.222 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2023-10-08T10:03:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.225 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.226 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2023-10-08T10:03:19.227 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.229 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.231 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.232 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.233 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.234 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T10:03:19.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2023-10-08T10:03:19.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.237 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T10:03:19.238 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.239 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2023-10-08T10:03:19.240 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.241 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.242 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.244 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.245 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.248 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.249 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2023-10-08T10:03:19.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.251 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.252 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.253 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.254 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.256 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.257 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.258 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.259 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.260 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.261 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.262 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.264 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.267 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "drive_group_interval": { 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "drive_group_interval", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300.0", 2023-10-08T10:03:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2023-10-08T10:03:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.286 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.287 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.288 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.291 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.292 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2023-10-08T10:03:19.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2023-10-08T10:03:19.295 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2023-10-08T10:03:19.298 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.299 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2023-10-08T10:03:19.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.302 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.303 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.304 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2023-10-08T10:03:19.305 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.306 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.307 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.308 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.309 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.310 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.313 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.314 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.315 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.316 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.317 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.318 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.319 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.320 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.321 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.322 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.326 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.327 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.328 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.329 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.330 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.331 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.332 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2023-10-08T10:03:19.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T10:03:19.336 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.337 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.338 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2023-10-08T10:03:19.339 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.341 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.342 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.344 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:19.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.347 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.349 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.350 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.351 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.356 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2023-10-08T10:03:19.357 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.360 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.361 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.364 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.367 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.368 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.370 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.372 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.373 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2023-10-08T10:03:19.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.377 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2023-10-08T10:03:19.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.379 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.380 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.381 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2023-10-08T10:03:19.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2023-10-08T10:03:19.384 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2023-10-08T10:03:19.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.387 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2023-10-08T10:03:19.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: "services": { 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard": "https://172.21.15.97:8443/" 2023-10-08T10:03:19.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "always_on_modules": { 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "octopus": [ 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2023-10-08T10:03:19.391 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "pacific": [ 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2023-10-08T10:03:19.392 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "quincy": [ 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2023-10-08T10:03:19.393 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "reef": [ 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2023-10-08T10:03:19.394 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "last_failure_osd_epoch": 3, 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "active_clients": [ 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-10-08T10:03:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2644603845 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "libcephsqlite", 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2446646431 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 4092341573 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3419108156 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-10-08T10:03:19.398 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T10:03:19.404 DEBUG:tasks.check_counter:Getting stats from a 2023-10-08T10:03:19.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-08T10:03:19.405 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:18 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3666642790' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T10:03:19.406 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:18 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3666642790' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T10:03:19.406 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:18 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3666642790' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T10:03:19.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:19 smithi178 ceph-mon[103156]: pgmap v545: 129 pgs: 129 active+clean; 5.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 167 op/s 2023-10-08T10:03:19.495 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:19 smithi097 ceph-mon[99491]: pgmap v545: 129 pgs: 129 active+clean; 5.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 167 op/s 2023-10-08T10:03:19.527 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:19 smithi186 ceph-mon[104393]: pgmap v545: 129 pgs: 129 active+clean; 5.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 151 KiB/s wr, 167 op/s 2023-10-08T10:03:20.455 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T10:03:21.704 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:21 smithi097 ceph-mon[99491]: pgmap v546: 129 pgs: 129 active+clean; 5.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 178 op/s 2023-10-08T10:03:21.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:21 smithi178 ceph-mon[103156]: pgmap v546: 129 pgs: 129 active+clean; 5.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 178 op/s 2023-10-08T10:03:21.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:21 smithi186 ceph-mon[104393]: pgmap v546: 129 pgs: 129 active+clean; 5.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 178 op/s 2023-10-08T10:03:22.845 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T10:03:22.846 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:22.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2023-10-08T10:03:22.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2023-10-08T10:03:22.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001182405, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T10:03:22.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000171643, 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:22.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:22.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.850 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:22.850 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 5, 2023-10-08T10:03:22.850 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 120, 2023-10-08T10:03:22.850 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3152, 2023-10-08T10:03:22.850 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 97728, 2023-10-08T10:03:22.851 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:22.851 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:22.851 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.008598250, 2023-10-08T10:03:22.851 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006076522, 2023-10-08T10:03:22.851 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001602090, 2023-10-08T10:03:22.852 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:22.852 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:22.852 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 119, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.026808384, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000225280 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T10:03:22.853 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 97728 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 158, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 157, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 183648, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 93232, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.021375301, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.009031018, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007343121, 2023-10-08T10:03:22.854 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.003709900, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 156, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.033681470, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000215906 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 183648, 2023-10-08T10:03:22.855 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 93232 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:22.856 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:22.857 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:22.858 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148984, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 30, 2023-10-08T10:03:22.859 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:22.860 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-10-08T10:03:22.861 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 159, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 161324, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 159, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 159, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 161324, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:22.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:22.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:22.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:22.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:22.863 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T10:03:23.234 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-10-08T10:03:23.234 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-10-08T10:03:23.234 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-10-08T10:03:23.234 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-10-08T10:03:23.234 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-10-08T10:03:23.234 DEBUG:tasks.check_counter:Getting stats from d 2023-10-08T10:03:23.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-08T10:03:23.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:23 smithi097 ceph-mon[99491]: pgmap v547: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 136 KiB/s wr, 220 op/s 2023-10-08T10:03:23.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:23 smithi178 ceph-mon[103156]: pgmap v547: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 136 KiB/s wr, 220 op/s 2023-10-08T10:03:23.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:23 smithi186 ceph-mon[104393]: pgmap v547: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 136 KiB/s wr, 220 op/s 2023-10-08T10:03:23.794 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T10:03:25.015 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2992, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 448, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001752251, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000046044, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000794532, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:25.016 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000870879, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000290293 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 162, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 161, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 184736, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 95728, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.022742619, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.009886151, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007149834, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.004060803, 2023-10-08T10:03:25.018 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:25.019 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 160, 2023-10-08T10:03:25.019 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.028964909, 2023-10-08T10:03:25.019 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000181030 2023-10-08T10:03:25.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.019 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:25.020 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:25.020 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:25.020 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:25.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.020 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 184736, 2023-10-08T10:03:25.021 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 95728 2023-10-08T10:03:25.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.021 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:25.022 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1, 2023-10-08T10:03:25.022 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 122, 2023-10-08T10:03:25.022 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 160, 2023-10-08T10:03:25.022 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 97984, 2023-10-08T10:03:25.022 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.009364281, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006959336, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001227970, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:25.023 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:25.024 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 120, 2023-10-08T10:03:25.024 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.043398002, 2023-10-08T10:03:25.024 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000361650 2023-10-08T10:03:25.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.024 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 97984 2023-10-08T10:03:25.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:25.026 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:25.027 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:25.028 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:25.029 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148967, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 29, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:25.030 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:25.031 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 163, 2023-10-08T10:03:25.032 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 161844, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 163, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 163, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 161844, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:25.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:25.034 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T10:03:25.444 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-08T10:03:25.445 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-08T10:03:25.445 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-10-08T10:03:25.445 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-10-08T10:03:25.445 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-10-08T10:03:25.445 DEBUG:tasks.check_counter:Getting stats from g 2023-10-08T10:03:25.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-08T10:03:25.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:25 smithi097 ceph-mon[99491]: pgmap v548: 129 pgs: 129 active+clean; 402 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 136 KiB/s wr, 305 op/s 2023-10-08T10:03:25.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:25 smithi178 ceph-mon[103156]: pgmap v548: 129 pgs: 129 active+clean; 402 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 136 KiB/s wr, 305 op/s 2023-10-08T10:03:25.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:25 smithi186 ceph-mon[104393]: pgmap v548: 129 pgs: 129 active+clean; 402 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 136 KiB/s wr, 305 op/s 2023-10-08T10:03:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T10:03:27.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:27 smithi097 ceph-mon[99491]: pgmap v549: 129 pgs: 129 active+clean; 214 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 262 op/s 2023-10-08T10:03:27.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:27 smithi178 ceph-mon[103156]: pgmap v549: 129 pgs: 129 active+clean; 214 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 262 op/s 2023-10-08T10:03:27.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:27 smithi186 ceph-mon[104393]: pgmap v549: 129 pgs: 129 active+clean; 214 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 262 op/s 2023-10-08T10:03:28.773 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T10:03:28.773 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:28.773 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 160, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 159, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 184192, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 94480, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.023873087, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.010000844, 2023-10-08T10:03:28.774 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007819362, 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.003846404, 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 158, 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.055119685, 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000348858 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.775 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 184192, 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 94480 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.776 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 120, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 160, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 97632, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.009597945, 2023-10-08T10:03:28.777 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006703609, 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001826720, 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 118, 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032638518, 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000276597 2023-10-08T10:03:28.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 97632 2023-10-08T10:03:28.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2992, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 448, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:28.780 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000772068, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000032881, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000713133, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000745445, 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000248481 2023-10-08T10:03:28.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:28.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:28.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:28.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:28.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148967, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 29, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-10-08T10:03:28.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2023-10-08T10:03:28.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-10-08T10:03:28.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 161, 2023-10-08T10:03:28.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 161584, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 161, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 161, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 161584, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:28.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:28.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:28.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:28.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:28.793 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T10:03:29.181 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-10-08T10:03:29.181 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-10-08T10:03:29.181 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-10-08T10:03:29.181 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-10-08T10:03:29.181 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-10-08T10:03:29.181 DEBUG:tasks.check_counter:Getting stats from j 2023-10-08T10:03:29.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-08T10:03:29.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:29 smithi097 ceph-mon[99491]: pgmap v550: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 KiB/s wr, 262 op/s 2023-10-08T10:03:29.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:29 smithi178 ceph-mon[103156]: pgmap v550: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 KiB/s wr, 262 op/s 2023-10-08T10:03:29.740 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T10:03:29.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:29 smithi186 ceph-mon[104393]: pgmap v550: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 KiB/s wr, 262 op/s 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 168, 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 165, 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 188272, 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 97424, 2023-10-08T10:03:30.996 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.022860298, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.009064070, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007458963, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.003971311, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 165, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.041759799, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000253089 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:30.997 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 188272, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 97424 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 123, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 160, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 98160, 2023-10-08T10:03:30.998 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.008794425, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006480899, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001164816, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 121, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.043761183, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000361662 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:30.999 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 98160 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2023-10-08T10:03:31.000 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000343298, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000270827, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:31.001 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-10-08T10:03:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:31.003 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:31.004 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-10-08T10:03:31.005 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 149024, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:31.006 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:31.007 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 165, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 162104, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 165, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 165, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 162104, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-10-08T10:03:31.009 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-10-08T10:03:31.402 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-08T10:03:31.402 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-08T10:03:31.402 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-08T10:03:31.402 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-08T10:03:31.402 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-10-08T10:03:31.402 DEBUG:tasks.check_counter:Getting stats from b 2023-10-08T10:03:31.402 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-08T10:03:31.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:31 smithi097 ceph-mon[99491]: pgmap v551: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 169 op/s 2023-10-08T10:03:31.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:31 smithi178 ceph-mon[103156]: pgmap v551: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 169 op/s 2023-10-08T10:03:31.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:31 smithi186 ceph-mon[104393]: pgmap v551: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 169 op/s 2023-10-08T10:03:32.136 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T10:03:33.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:33 smithi186 ceph-mon[104393]: pgmap v552: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 159 op/s 2023-10-08T10:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 4302, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 5400, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1253370, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 10880664, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 40, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.342561474, 2023-10-08T10:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.105748709, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.133858382, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.090166109, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5401, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.478729012, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000273788 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4326, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000751380, 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000173 2023-10-08T10:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 2806, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3469, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1408350, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 3358046, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 9, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2023-10-08T10:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.213291613, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.066752606, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.088869083, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.055806134, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3469, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.649908397, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000475614 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2860, 2023-10-08T10:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000530671, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000185 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 342496, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 109616 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 7517, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 11675, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 2639677, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 7883556, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 39, 2023-10-08T10:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 8, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.615585827, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.233288239, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.446639638, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.075641436, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 11669, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.251187577, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000192920 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 12663, 2023-10-08T10:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002166842, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000171 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2189120, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 6818816 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2636, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.149255541, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000815347 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2394, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.165357394, 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000069071 2023-10-08T10:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 2410, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 2410, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2410, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 14.526067119, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.006027413 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2023-10-08T10:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 2, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 143, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 31, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 9, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 9, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 13, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 17, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 351, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 1, 2023-10-08T10:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 1, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 3425, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 3148, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 58, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 3281, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 11, 2023-10-08T10:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 326, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 7, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 303, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 401, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 401, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 104, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 4, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 100, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 297, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 2, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 593, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 2768, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 6, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 3014, 2023-10-08T10:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 1935, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 33, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 1278 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 169, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 340, 2023-10-08T10:03:33.852 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 6, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 842, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 775, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 61, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2023-10-08T10:03:33.853 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 4, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 11, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 5193, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 4440, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 4440, 2023-10-08T10:03:33.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 753, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 6, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 18, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 17, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 17, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 1, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 1, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 1, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 12830169, 2023-10-08T10:03:33.855 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 14252063, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1201, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 78.964214566, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.065748721 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 21, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 1034, 2023-10-08T10:03:33.856 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 1013, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 20, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 189, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 169, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 31, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 1733, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 1702, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 1, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 1079, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 1078, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 64848, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 207152 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.857 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 3380, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 3, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 2410, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 20, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 11, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 686, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.600670538, 2023-10-08T10:03:33.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000875613 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 40, 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.011807604, 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000295190 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:33.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 61, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.050059453, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000820646 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 334, 2023-10-08T10:03:33.860 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.491544543, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001471690 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:33.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 99, 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.080353158, 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000811648 2023-10-08T10:03:33.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.865 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2023-10-08T10:03:33.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 73, 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.054784065, 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000750466 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 172, 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.726354055, 2023-10-08T10:03:33.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004222988 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 111, 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 9.881285878, 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.089020593 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:33.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 97, 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.618350471, 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.016684025 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:33.870 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 7, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.026579921, 2023-10-08T10:03:33.871 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.003797131 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4, 2023-10-08T10:03:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000931848, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000232962 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002711615, 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002711615 2023-10-08T10:03:33.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 13, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.008518510, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000655270 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 712, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.972115460, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001365330 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:33.874 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2023-10-08T10:03:33.875 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 500, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 500, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 466, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:33.876 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:33.877 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:33.878 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:33.879 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:33.880 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:33.881 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 275155, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 122, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 880, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 10, 2023-10-08T10:03:33.882 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 168, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:33.883 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 98549, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 991, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 5378, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 10897180, 2023-10-08T10:03:33.884 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 37, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 5378, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5378, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 6074.236541885, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 1.129460123 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5378, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 7016 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 5378, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 438, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 4940, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 51, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 873, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 31, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 1228, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 79, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2023-10-08T10:03:33.886 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 2578, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 407, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 1359, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:33.887 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 6, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2023-10-08T10:03:33.888 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2023-10-08T10:03:33.889 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 9, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 317, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 4, 2023-10-08T10:03:33.890 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 83 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2023-10-08T10:03:33.891 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 52, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 16, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 775, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:33.892 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 14621, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 3710907, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 14621, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 14621, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 3710907, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.893 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:33.894 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5378, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 10980791, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5378, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10980791, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:33.895 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5378, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 5378, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5378, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 5378, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.896 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 775, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 83643, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 775, 2023-10-08T10:03:33.897 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 27, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 83643, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:33.898 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x561a7d7586e0": { 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 5193, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 10057759, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 5193, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1201, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10057759, 2023-10-08T10:03:33.899 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:33.900 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-08T10:03:33.900 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:33 smithi178 ceph-mon[103156]: pgmap v552: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 159 op/s 2023-10-08T10:03:33.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:33 smithi097 ceph-mon[99491]: pgmap v552: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 159 op/s 2023-10-08T10:03:34.185 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2023-10-08T10:03:34.185 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2023-10-08T10:03:34.185 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=169 2023-10-08T10:03:34.186 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-10-08T10:03:34.186 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2023-10-08T10:03:34.186 DEBUG:tasks.check_counter:Getting stats from e 2023-10-08T10:03:34.186 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-08T10:03:34.743 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T10:03:35.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:35 smithi178 ceph-mon[103156]: pgmap v553: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 115 op/s 2023-10-08T10:03:35.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:35 smithi186 ceph-mon[104393]: pgmap v553: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 115 op/s 2023-10-08T10:03:35.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:35 smithi097 ceph-mon[99491]: pgmap v553: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 115 op/s 2023-10-08T10:03:35.960 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-08T10:03:35.960 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 203, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 174, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 318050, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 99453, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.026229931, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009425431, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.010287364, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.004892072, 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:35.961 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 173, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.033389637, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000193003 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 317216, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 99136 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.962 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 374, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 496, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 91940, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 397400, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.044253338, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.012781611, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.010597267, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.016847352, 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:35.963 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 494, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.147480317, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000298543 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 281392 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.964 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1240, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1240, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3005786, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 384902, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.130359918, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.037242169, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.035792789, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.047031842, 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:35.965 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1239, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.218642911, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000176467 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.966 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2826, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.079477523, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000028123 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.967 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2023-10-08T10:03:35.968 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 39, 2023-10-08T10:03:35.969 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 39, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 39, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2023-10-08T10:03:35.970 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:35.971 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2023-10-08T10:03:35.972 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2023-10-08T10:03:35.973 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2023-10-08T10:03:35.974 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.975 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 2000, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2023-10-08T10:03:35.976 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 3, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 6837505, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 6835854, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 2000 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 4, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 252, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 248, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 2, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 89, 2023-10-08T10:03:35.978 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 87, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 39, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 368, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 329, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2023-10-08T10:03:35.979 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 51108, 2023-10-08T10:03:35.980 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 207152 2023-10-08T10:03:35.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.980 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2023-10-08T10:03:35.980 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2023-10-08T10:03:35.980 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.981 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2023-10-08T10:03:35.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.985 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.986 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.987 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.989 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.990 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.991 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.992 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:35.994 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 1, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1836, 2023-10-08T10:03:35.995 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:35.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:35.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:35.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 190247, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 54, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:35.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 168, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 39248, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 184, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-08T10:03:36.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 1610, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 1610, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1610, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 31.032428070, 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.019274800 2023-10-08T10:03:36.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1610, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1611 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 1610, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 1610, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 894, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2023-10-08T10:03:36.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 714, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2023-10-08T10:03:36.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2023-10-08T10:03:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2023-10-08T10:03:36.005 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 4, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2023-10-08T10:03:36.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T10:03:36.007 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:36.009 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1813, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 3239361, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1813, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1813, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 3239361, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:36.010 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:36.011 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1610, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 2643206, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1610, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2643206, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:36.012 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1610, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1610, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1610, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1610, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:36.013 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:36.014 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x563ed0332960": { 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:36.015 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:36.016 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.017 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:36.017 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-08T10:03:36.385 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-08T10:03:36.385 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-08T10:03:36.385 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-10-08T10:03:36.385 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-10-08T10:03:36.385 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-10-08T10:03:36.385 DEBUG:tasks.check_counter:Getting stats from h 2023-10-08T10:03:36.386 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-08T10:03:36.947 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T10:03:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:37 smithi178 ceph-mon[103156]: pgmap v554: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 22 op/s 2023-10-08T10:03:37.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:37 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:03:37.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:37 smithi186 ceph-mon[104393]: pgmap v554: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 22 op/s 2023-10-08T10:03:37.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:37 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:03:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:37 smithi097 ceph-mon[99491]: pgmap v554: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 22 op/s 2023-10-08T10:03:37.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:37 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 689, 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 688, 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 4414973, 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 213182, 2023-10-08T10:03:38.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.071307735, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.024500476, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.017953995, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.021944602, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 687, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.117294518, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000170734 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 667, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 638, 2023-10-08T10:03:38.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 2102773, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 243353, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.063516740, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.017423727, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.021410828, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.021313226, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 637, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.106005481, 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000166413 2023-10-08T10:03:38.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 317104, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 98672 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 593, 2023-10-08T10:03:38.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 715, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 4345273, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 467089, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.063187464, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.016725011, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.019749108, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.021356091, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 713, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.133392278, 2023-10-08T10:03:38.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000187085 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 282976 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.178 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2939, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.087652484, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000029823 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.179 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2023-10-08T10:03:38.180 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 471, 2023-10-08T10:03:38.181 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 2, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 6, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 463, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 468, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 574, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2023-10-08T10:03:38.182 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2023-10-08T10:03:38.183 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T10:03:38.184 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.185 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2023-10-08T10:03:38.186 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2023-10-08T10:03:38.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 5193, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2023-10-08T10:03:38.188 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 14, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 12830169, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 14252063, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 14250925, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.189 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 5193 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 13, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 1029, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 1016, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 12, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 182, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 170, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 471, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 1665, 2023-10-08T10:03:38.190 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 1194, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 49968, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 207152 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2023-10-08T10:03:38.191 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:38.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2023-10-08T10:03:38.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.198 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.200 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2023-10-08T10:03:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1837, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-08T10:03:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189667, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 58, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 168, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 377273, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2383, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 1741, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 1741, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1741, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 48.992101214, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.028140207 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1741, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1742 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 1741, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 1741, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1012, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 726, 2023-10-08T10:03:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2023-10-08T10:03:38.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2023-10-08T10:03:38.214 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2023-10-08T10:03:38.215 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 4, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2023-10-08T10:03:38.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:38.217 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1945, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 10674730, 2023-10-08T10:03:38.218 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1945, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1945, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10674730, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.219 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1741, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 10058619, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1741, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 10058619, 2023-10-08T10:03:38.220 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:38.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1741, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1741, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1741, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1741, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2023-10-08T10:03:38.223 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x56106e2b0960": { 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2023-10-08T10:03:38.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:38.225 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-08T10:03:38.508 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-10-08T10:03:38.508 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-10-08T10:03:38.509 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-10-08T10:03:38.509 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-10-08T10:03:38.509 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-10-08T10:03:38.509 DEBUG:tasks.check_counter:Getting stats from k 2023-10-08T10:03:38.509 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-08T10:03:38.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:38 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:38 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:38.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:38 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:39.068 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T10:03:39.568 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:39 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi097 to 196.2M 2023-10-08T10:03:39.568 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:39 smithi178 ceph-mon[103156]: Unable to set osd_memory_target on smithi097 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-10-08T10:03:39.569 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:39 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi178 to 196.2M 2023-10-08T10:03:39.569 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:39 smithi178 ceph-mon[103156]: Unable to set osd_memory_target on smithi178 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-10-08T10:03:39.569 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:39 smithi178 ceph-mon[103156]: pgmap v555: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 16 op/s 2023-10-08T10:03:39.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:39 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi097 to 196.2M 2023-10-08T10:03:39.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:39 smithi186 ceph-mon[104393]: Unable to set osd_memory_target on smithi097 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-10-08T10:03:39.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:39 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi178 to 196.2M 2023-10-08T10:03:39.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:39 smithi186 ceph-mon[104393]: Unable to set osd_memory_target on smithi178 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-10-08T10:03:39.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:39 smithi186 ceph-mon[104393]: pgmap v555: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 16 op/s 2023-10-08T10:03:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:39 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi097 to 196.2M 2023-10-08T10:03:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:39 smithi097 ceph-mon[99491]: Unable to set osd_memory_target on smithi097 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-10-08T10:03:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:39 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi178 to 196.2M 2023-10-08T10:03:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:39 smithi097 ceph-mon[99491]: Unable to set osd_memory_target on smithi178 to 205738086: error parsing value: Value '205738086' is below minimum 939524096 2023-10-08T10:03:39.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:39 smithi097 ceph-mon[99491]: pgmap v555: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 16 op/s 2023-10-08T10:03:40.326 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-08T10:03:40.326 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 4, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 4, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 2992, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 448, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.001559401, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000037223, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000734907, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:40.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:40.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2023-10-08T10:03:40.328 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000746900, 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000248966 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 126, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 160, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 98688, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.007045701, 2023-10-08T10:03:40.330 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.004888100, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000969281, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 124, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.028292520, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000228165 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:40.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 98688 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 167, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 166, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 186096, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 98848, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:40.332 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.024584183, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009624763, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.007715764, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005085509, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 165, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.023541973, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000142678 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:40.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 186096, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 98848 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:40.334 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:40.335 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:40.336 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:40.337 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 149047, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 31, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2023-10-08T10:03:40.338 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-08T10:03:40.339 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 168, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 162494, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 168, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-08T10:03:40.340 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 168, 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 162494, 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-08T10:03:40.341 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-08T10:03:40.628 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-10-08T10:03:40.629 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-10-08T10:03:40.629 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-10-08T10:03:40.629 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-10-08T10:03:40.629 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-10-08T10:03:40.629 DEBUG:tasks.check_counter:Getting stats from c 2023-10-08T10:03:40.629 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-08T10:03:41.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: pgmap v556: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:41.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:41.549 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:41.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:41.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:41.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:03:41.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:03:41.550 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:41 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:03:41.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: pgmap v556: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:42.018 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.018 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:03:42.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:03:42.019 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:41 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:03:42.020 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T10:03:42.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: pgmap v556: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:42.020 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.021 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T10:03:42.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T10:03:42.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T10:03:42.022 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:41 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' 2023-10-08T10:03:42.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:42 smithi186 ceph-mon[104393]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-08T10:03:42.792 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:42 smithi186 ceph-mon[104393]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-08T10:03:42.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:42 smithi097 ceph-mon[99491]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-08T10:03:43.023 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:42 smithi097 ceph-mon[99491]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-08T10:03:43.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:42 smithi178 ceph-mon[103156]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-08T10:03:43.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:42 smithi178 ceph-mon[103156]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-08T10:03:43.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:43 smithi186 ceph-mon[104393]: pgmap v557: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:43.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:43 smithi097 ceph-mon[99491]: pgmap v557: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:43.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:43 smithi178 ceph-mon[103156]: pgmap v557: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3413, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 3658, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1383283, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 4282743, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 18, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.270594904, 2023-10-08T10:03:45.463 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.095293687, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.212221972, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.038206716, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3656, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.679602117, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000459409 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3438, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000679688, 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000197 2023-10-08T10:03:45.464 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1153424, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 1283072 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 803820, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 803848, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 130522347, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 127563593, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 37.208363910, 2023-10-08T10:03:45.465 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 16.042938214, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 26.987949806, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.019061770, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 803848, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 19.650236320, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000024445 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1606328, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.212263089, 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000132 2023-10-08T10:03:45.466 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 286528 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2250, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1566, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1010074, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 687725, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 10, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 8, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.140960789, 2023-10-08T10:03:45.467 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.046388676, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.099095954, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.017254467, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1564, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.276340828, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000176688 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3025, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000586862, 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000194 2023-10-08T10:03:45.468 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 916, 2023-10-08T10:03:45.469 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.554279071, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000605108 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1071, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.040459972, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000037777 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.470 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 288, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 269, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 269, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.296451623, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001102050 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 19, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:45.471 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 12, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 2, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 10, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 10, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 334, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2023-10-08T10:03:45.472 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 651, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 534, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 19, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 98, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 7, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 280, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 11, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 318, 2023-10-08T10:03:45.473 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 187, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 187, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 63, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T10:03:45.474 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 63, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 124, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 7, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 494, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 1876, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 1226, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 15, 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 78 2023-10-08T10:03:45.475 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 273, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 181, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 192, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 38, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 38, 2023-10-08T10:03:45.476 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 7, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:45.477 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 2000, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 2000, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 3, 2023-10-08T10:03:45.478 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 3, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 6837505, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 370, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 11.574415624, 2023-10-08T10:03:45.479 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.031282204 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 17, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 283, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 266, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 15, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 129, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 114, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 12, 2023-10-08T10:03:45.480 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 422, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 410, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 324, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 324, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 57004, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 198960 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 288, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 2, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 288, 2023-10-08T10:03:45.481 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 9, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 6, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.482 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.483 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 171, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.180579629, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001056021 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.484 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.485 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.486 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.487 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 58, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.079829292, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001376367 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001590317, 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000795158 2023-10-08T10:03:45.488 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:45.489 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.490 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:45.491 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.492 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.493 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 38, 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.034452385, 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000906641 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:45.494 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:45.495 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.495 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 109, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 109, 2023-10-08T10:03:45.496 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 409, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:45.497 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:45.498 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-08T10:03:45.499 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 284823, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 119, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:45.500 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 168, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 65192, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 248, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:45.501 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 1808, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 2701854, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 28, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 1808, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1808, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 129.563627376, 2023-10-08T10:03:45.502 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.071661298 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1808, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1946 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 1808, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 188, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 1620, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-08T10:03:45.503 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 371, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 30, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 1130, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:45.504 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 187, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-10-08T10:03:45.505 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-08T10:03:45.506 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 89, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 4, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 12, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 134, 2023-10-08T10:03:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 63 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.508 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 52, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 1, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 38, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 809479, 2023-10-08T10:03:45.509 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 59950101, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 809479, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 809479, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 59950101, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.510 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1808, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2705996, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1808, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2705996, 2023-10-08T10:03:45.511 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:45.512 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1808, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1808, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1808, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1808, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:45.513 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 38, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 4142, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 38, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 4142, 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:45.514 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x563dcdd5e6e0": { 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2000, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2643201, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:45.515 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2000, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 370, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2643201, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:45.516 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-08T10:03:45.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:45 smithi186 ceph-mon[104393]: pgmap v558: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:45.873 INFO:tasks.check_counter:Daemon mds.c mds.exported=7 2023-10-08T10:03:45.873 INFO:tasks.check_counter:Daemon mds.c mds.imported=11 2023-10-08T10:03:45.873 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-10-08T10:03:45.873 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=273 2023-10-08T10:03:45.873 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-10-08T10:03:45.873 DEBUG:tasks.check_counter:Getting stats from f 2023-10-08T10:03:45.873 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-08T10:03:45.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:45 smithi097 ceph-mon[99491]: pgmap v558: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:45.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:45 smithi178 ceph-mon[103156]: pgmap v558: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:46.434 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1607, 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1609, 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 5474462, 2023-10-08T10:03:47.614 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 499784, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.145397731, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.038394353, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.039051926, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.057125028, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1607, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.276365816, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000171976 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.615 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 210, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 179, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 321308, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 101516, 2023-10-08T10:03:47.616 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.026962342, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.009520587, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.011444968, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005286467, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 179, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.031491651, 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000175931 2023-10-08T10:03:47.617 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 320752, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 100832 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:47.618 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 2, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 126, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 428, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 285223, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.007768268, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.004767291, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001684545, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000103787, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 124, 2023-10-08T10:03:47.619 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.020495072, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000165282 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 284912 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.620 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2757, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.075808860, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000027496 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.621 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-08T10:03:47.622 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 53, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2023-10-08T10:03:47.623 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 53, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 53, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 7, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:47.624 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:47.625 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2023-10-08T10:03:47.626 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T10:03:47.627 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2023-10-08T10:03:47.627 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T10:03:47.627 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2023-10-08T10:03:47.628 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-08T10:03:47.629 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2023-10-08T10:03:47.630 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 2414, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 4, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2023-10-08T10:03:47.631 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 9217085, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 9215434, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 2414 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 4, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 173, 2023-10-08T10:03:47.632 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 169, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 2, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 21, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 19, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 53, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 229, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 176, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 51452, 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207152 2023-10-08T10:03:47.633 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.634 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.635 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.636 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.637 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.638 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-08T10:03:47.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:47.641 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:47.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.644 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.645 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2023-10-08T10:03:47.647 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 1857, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:47.648 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:47.649 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-08T10:03:47.650 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 181343, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 47, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:47.651 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 168, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 49104, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 240, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-08T10:03:47.652 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 1610, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 1610, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1610, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 66.754750892, 2023-10-08T10:03:47.653 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.041462578 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1610, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1611 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 1610, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 1610, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-08T10:03:47.654 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 896, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 712, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-08T10:03:47.655 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-08T10:03:47.656 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-08T10:03:47.657 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 89, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 4, 2023-10-08T10:03:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 4, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 4, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-08T10:03:47.659 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.660 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1815, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 5619499, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1815, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1815, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5619499, 2023-10-08T10:03:47.661 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:47.662 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1610, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 5022786, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1610, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5022786, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.663 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 1610, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 1610, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1610, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1610, 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:47.664 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:47.665 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x557334f40960": { 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:47.666 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:47.667 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:47.668 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:47.668 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.668 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:47.668 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-08T10:03:47.742 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:47 smithi186 ceph-mon[104393]: pgmap v559: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:47.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:47 smithi097 ceph-mon[99491]: pgmap v559: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:47.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:47 smithi178 ceph-mon[103156]: pgmap v559: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:47.979 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-10-08T10:03:47.979 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-10-08T10:03:47.979 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-10-08T10:03:47.979 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-10-08T10:03:47.979 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-10-08T10:03:47.979 DEBUG:tasks.check_counter:Getting stats from i 2023-10-08T10:03:47.979 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-08T10:03:48.535 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T10:03:49.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:49 smithi186 ceph-mon[104393]: pgmap v560: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:49.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:49 smithi097 ceph-mon[99491]: pgmap v560: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:49.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:49 smithi178 ceph-mon[103156]: pgmap v560: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:51.294 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-08T10:03:51.294 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4494, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 3406, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1522756, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 6248086, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 11, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.354812322, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.105165443, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.163174109, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.108589353, 2023-10-08T10:03:51.295 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3404, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.653016982, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000191838 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4529, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000762917, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000168 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:51.296 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 803812, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 803876, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 127481775, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 130660575, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-10-08T10:03:51.297 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 35.718225123, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 15.679790576, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 25.435332296, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.018467303, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 803875, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 26.423484918, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000032870 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1606287, 2023-10-08T10:03:51.298 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.193646265, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000120 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 331040, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 106736 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 3562, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 4843, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1210295, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 2366475, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 17, 2023-10-08T10:03:51.299 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 7, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.297703782, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.111739308, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.227270996, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.031854702, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4842, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.740573243, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000359474 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:51.300 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4677, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001200061, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000256 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 963760, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 1953808 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-08T10:03:51.301 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2096, 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.045869822, 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000498983 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.302 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1489, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.053766730, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000036109 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 325, 2023-10-08T10:03:51.303 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 312, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 312, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.413582116, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001325583 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 13, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 20, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 1, 2023-10-08T10:03:51.304 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 13, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 1, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 2, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 10, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 10, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 281, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 2, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 633, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 530, 2023-10-08T10:03:51.305 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 13, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 82, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 7, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 110, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 7, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 95, 2023-10-08T10:03:51.306 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 77, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 77, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 2, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T10:03:51.307 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 2, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 75, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 4, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 13, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 1235, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 2217, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 1822, 2023-10-08T10:03:51.308 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 13, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 107 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 4, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 243, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 162, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 160, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 12, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-08T10:03:51.309 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 52, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 52, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T10:03:51.310 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 7, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 1, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 2414, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2023-10-08T10:03:51.311 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 2414, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 4, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 4, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 9217085, 2023-10-08T10:03:51.312 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1139, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 62.202038457, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.054611096 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 17, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 264, 2023-10-08T10:03:51.313 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 247, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 15, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 116, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 101, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 13, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 404, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 391, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 238, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 238, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 54868, 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 198960 2023-10-08T10:03:51.314 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 347, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 4, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 325, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 13, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 14, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 73, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.104215380, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001427607 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.315 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.316 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 136, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.098245288, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000722391 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.317 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-08T10:03:51.318 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.319 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.320 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 46, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.077103092, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001676154 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-08T10:03:51.321 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.003124213, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000624842 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.322 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.053253575, 2023-10-08T10:03:51.323 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.026626787 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.324 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-08T10:03:51.325 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.326 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 50, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.077640568, 2023-10-08T10:03:51.327 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001552811 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 133, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 133, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 413, 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-08T10:03:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:51.331 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-08T10:03:51.332 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 292279, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 120, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:51.333 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8792, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 168, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 66392, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 274, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-08T10:03:51.334 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2936, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 5231195, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 27, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2936, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2936, 2023-10-08T10:03:51.335 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 233.228695547, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.079437566 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2936, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 3079 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2936, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 79, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 2857, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-08T10:03:51.336 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 2, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 1141, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 32, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 1, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 1583, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-08T10:03:51.337 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 77, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-08T10:03:51.338 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-08T10:03:51.339 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 89, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 4, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 12, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-08T10:03:51.340 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 150, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 63 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.341 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 52, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 2, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 52, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.342 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 811864, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 57018699, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 811864, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 811864, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 57018699, 2023-10-08T10:03:51.343 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:51.344 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2936, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 5236855, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2936, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5236855, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.345 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2936, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2936, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2936, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2936, 2023-10-08T10:03:51.346 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 52, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 5660, 2023-10-08T10:03:51.347 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 52, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5660, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:51.348 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x564cb73866e0": { 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2414, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 5022781, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2414, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1139, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5022781, 2023-10-08T10:03:51.349 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:51.350 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-08T10:03:51.646 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:51 smithi186 ceph-mon[104393]: pgmap v561: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:51.689 INFO:tasks.check_counter:Daemon mds.i mds.exported=7 2023-10-08T10:03:51.689 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2023-10-08T10:03:51.689 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4 2023-10-08T10:03:51.690 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=243 2023-10-08T10:03:51.690 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2023-10-08T10:03:51.690 DEBUG:tasks.check_counter:Getting stats from l 2023-10-08T10:03:51.690 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-08T10:03:51.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:51 smithi097 ceph-mon[99491]: pgmap v561: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:51.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:51 smithi178 ceph-mon[103156]: pgmap v561: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-08T10:03:52.249 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 171, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 170, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 187184, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 101344, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-08T10:03:53.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.024909911, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.009312258, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.007785994, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005044919, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 169, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.045201672, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000267465 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:53.454 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 187184, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 101344 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 5, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 131, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3152, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 99664, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.007509436, 2023-10-08T10:03:53.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.004922469, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001331912, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 130, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.026374165, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000202878 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:53.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 99664 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000361570, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T10:03:53.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000294480, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T10:03:53.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-08T10:03:53.459 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-08T10:03:53.460 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-08T10:03:53.461 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148984, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 30, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-08T10:03:53.462 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-08T10:03:53.463 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 172, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 163014, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 172, 2023-10-08T10:03:53.464 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 172, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 163014, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-08T10:03:53.465 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-08T10:03:53.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:53 smithi186 ceph-mon[104393]: pgmap v562: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:53.804 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-10-08T10:03:53.804 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-10-08T10:03:53.804 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-10-08T10:03:53.804 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-10-08T10:03:53.804 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-10-08T10:03:53.805 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T10:03:53.815 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T10:03:53.816 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T10:03:53.816 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T10:03:53.845 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T10:03:53.845 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T10:03:53.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T10:03:53.952 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:53 smithi097 ceph-mon[99491]: pgmap v562: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:53.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:53 smithi178 ceph-mon[103156]: pgmap v562: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-10-08T10:03:54.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 -- 172.21.15.97:0/3737533684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a80100790 msgr2=0x7f7a80100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 --2- 172.21.15.97:0/3737533684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a80100790 0x7f7a80100bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a700099c0 tx=0x7f7a70009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 -- 172.21.15.97:0/3737533684 shutdown_connections 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 --2- 172.21.15.97:0/3737533684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 0x7f7a801079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 --2- 172.21.15.97:0/3737533684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a800ff5a0 0x7f7a800ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 --2- 172.21.15.97:0/3737533684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a80100790 0x7f7a80100bf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 -- 172.21.15.97:0/3737533684 >> 172.21.15.97:0/3737533684 conn(0x7f7a800fab10 msgr2=0x7f7a800fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 -- 172.21.15.97:0/3737533684 shutdown_connections 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.079+0000 7f7a84d01700 1 -- 172.21.15.97:0/3737533684 wait complete. 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 Processor -- start 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 -- start start 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 0x7f7a8019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 0x7f7a8019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e0a0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e210 con 0x7f7a80100790 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.080+0000 7f7a84d01700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e3d0 con 0x7f7a80101130 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58690/0 (socket says 172.21.15.97:58690) 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 -- 172.21.15.97:0/529985091 learned_addr learned my addr 172.21.15.97:0/529985091 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7ed9d700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 0x7f7a8019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7dd9b700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 0x7f7a8019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 -- 172.21.15.97:0/529985091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 msgr2=0x7f7a8019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 0x7f7a8019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 -- 172.21.15.97:0/529985091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 msgr2=0x7f7a8019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 0x7f7a8019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 -- 172.21.15.97:0/529985091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700095f0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7dd9b700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 0x7f7a8019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a7e59c700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e970 tx=0x7f7a6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68009790 con 0x7f7a800ff5a0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6800eec0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8019e6b0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68003cf0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.081+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8019ebc0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.082+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a801069d0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.084+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7a6800ccd0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.084+0000 7f7a777fe700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 0x7f7a6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.085+0000 7f7a7dd9b700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 0x7f7a6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.085+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7a68014070 con 0x7f7a800ff5a0 2023-10-08T10:03:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.085+0000 7f7a7dd9b700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 0x7f7a6c0a3940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006010 tx=0x7f7a70019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.088+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7a68096760 con 0x7f7a800ff5a0 2023-10-08T10:03:54.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.281+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a8010adc0 con 0x7f7a800ff5a0 2023-10-08T10:03:54.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.282+0000 7f7a777fe700 1 -- 172.21.15.97:0/529985091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f7a68099d80 con 0x7f7a800ff5a0 2023-10-08T10:03:54.284 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:54.285 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T10:03:54.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 msgr2=0x7f7a6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 0x7f7a6c0a3940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006010 tx=0x7f7a70019040 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 msgr2=0x7f7a80068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800e970 tx=0x7f7a6800ec80 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 shutdown_connections 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f7a6c0a1480 0x7f7a6c0a3940 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a80101130 0x7f7a8019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800ff5a0 0x7f7a80068350 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 --2- 172.21.15.97:0/529985091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a80100790 0x7f7a8019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.284+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 >> 172.21.15.97:0/529985091 conn(0x7f7a800fab10 msgr2=0x7f7a80103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.285+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 shutdown_connections 2023-10-08T10:03:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.285+0000 7f7a84d01700 1 -- 172.21.15.97:0/529985091 wait complete. 2023-10-08T10:03:54.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:50:14.128447+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:53:54.548005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:53:55.793692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:53:59.455547+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T10:03:54.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T10:03:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 -- 172.21.15.97:0/996285785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce40ff6a0 msgr2=0x7f9ce40ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 --2- 172.21.15.97:0/996285785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce40ffac0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40099c0 tx=0x7f9cd4009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 -- 172.21.15.97:0/996285785 shutdown_connections 2023-10-08T10:03:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 --2- 172.21.15.97:0/996285785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce4101230 0x7f9ce4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 --2- 172.21.15.97:0/996285785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce40ffac0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 --2- 172.21.15.97:0/996285785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 0x7f9ce4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 -- 172.21.15.97:0/996285785 >> 172.21.15.97:0/996285785 conn(0x7f9ce40fac10 msgr2=0x7f9ce40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 -- 172.21.15.97:0/996285785 shutdown_connections 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.461+0000 7f9cea33c700 1 -- 172.21.15.97:0/996285785 wait complete. 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 Processor -- start 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 -- start start 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce4103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 0x7f9ce4101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4102a40 con 0x7f9ce4101230 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4102bb0 con 0x7f9ce40ff6a0 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9cea33c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41a0830 con 0x7f9ce4100890 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9ce88d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9ce88d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58708/0 (socket says 172.21.15.97:58708) 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9ce88d9700 1 -- 172.21.15.97:0/3855227063 learned_addr learned my addr 172.21.15.97:0/3855227063 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9ce37fe700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 0x7f9ce4101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.462+0000 7f9ce3fff700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce4103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 -- 172.21.15.97:0/3855227063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 msgr2=0x7f9ce4101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 0x7f9ce4101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 -- 172.21.15.97:0/3855227063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce40ff6a0 msgr2=0x7f9ce4103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce4103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 -- 172.21.15.97:0/3855227063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40095f0 con 0x7f9ce4101230 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce88d9700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc007ee0 tx=0x7f9cdc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00dd20 con 0x7f9ce4101230 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc00f040 con 0x7f9ce4101230 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0048a0 con 0x7f9ce4101230 2023-10-08T10:03:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce41a0a90 con 0x7f9ce4101230 2023-10-08T10:03:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.463+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41a0f50 con 0x7f9ce4101230 2023-10-08T10:03:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.464+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4067530 con 0x7f9ce4101230 2023-10-08T10:03:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.465+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9cdc003a00 con 0x7f9ce4101230 2023-10-08T10:03:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.465+0000 7f9ce17fa700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 0x7f9ccc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.465+0000 7f9ce3fff700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 0x7f9ccc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.466+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9cdc0cd0a0 con 0x7f9ce4101230 2023-10-08T10:03:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.466+0000 7f9ce3fff700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 0x7f9ccc0a39c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4003730 tx=0x7f9cd4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.469+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9cdc096370 con 0x7f9ce4101230 2023-10-08T10:03:54.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.662+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ce410aec0 con 0x7f9ce4101230 2023-10-08T10:03:54.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.662+0000 7f9ce17fa700 1 -- 172.21.15.97:0/3855227063 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cdc0230a0 con 0x7f9ce4101230 2023-10-08T10:03:54.666 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 msgr2=0x7f9ccc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 0x7f9ccc0a39c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4003730 tx=0x7f9cd4000bc0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 msgr2=0x7f9ce4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc007ee0 tx=0x7f9cdc00d3b0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 shutdown_connections 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f9ccc0a1500 0x7f9ccc0a39c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce4101230 0x7f9ce4102340 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ce40ff6a0 0x7f9ce4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 --2- 172.21.15.97:0/3855227063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce4100890 0x7f9ce4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 >> 172.21.15.97:0/3855227063 conn(0x7f9ce40fac10 msgr2=0x7f9ce4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:54.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 shutdown_connections 2023-10-08T10:03:54.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.665+0000 7f9cea33c700 1 -- 172.21.15.97:0/3855227063 wait complete. 2023-10-08T10:03:54.678 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:54 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/529985091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:54.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T10:03:54.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:54 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/529985091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 -- 172.21.15.97:0/949803258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 msgr2=0x7f4800100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 --2- 172.21.15.97:0/949803258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f4800100d00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47f00099c0 tx=0x7f47f0009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 -- 172.21.15.97:0/949803258 shutdown_connections 2023-10-08T10:03:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 --2- 172.21.15.97:0/949803258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 0x7f4800107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 --2- 172.21.15.97:0/949803258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f48000ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 --2- 172.21.15.97:0/949803258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f4800100d00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 -- 172.21.15.97:0/949803258 >> 172.21.15.97:0/949803258 conn(0x7f48000fac40 msgr2=0x7f48000fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 -- 172.21.15.97:0/949803258 shutdown_connections 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.801+0000 7f4805b65700 1 -- 172.21.15.97:0/949803258 wait complete. 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 Processor -- start 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 -- start start 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f4800069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 0x7f480019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480019e1a0 con 0x7f48001008a0 2023-10-08T10:03:54.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480019e310 con 0x7f48000ff6b0 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f4805b65700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480019e4d0 con 0x7f4800101240 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58720/0 (socket says 172.21.15.97:58720) 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 -- 172.21.15.97:0/1513619347 learned_addr learned my addr 172.21.15.97:0/1513619347 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47fffff700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 0x7f480019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47ff7fe700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f4800069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 -- 172.21.15.97:0/1513619347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 msgr2=0x7f480019da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 0x7f480019da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 -- 172.21.15.97:0/1513619347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 msgr2=0x7f4800069810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.802+0000 7f47feffd700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f4800069810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:54.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47feffd700 1 -- 172.21.15.97:0/1513619347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00095f0 con 0x7f48001008a0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47ff7fe700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f4800069810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47feffd700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009fd0 tx=0x7f47f0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f001d070 con 0x7f48001008a0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47f00059a0 con 0x7f48001008a0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480019e750 con 0x7f48001008a0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00173f0 con 0x7f48001008a0 2023-10-08T10:03:54.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.803+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480019ec00 con 0x7f48001008a0 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.804+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4800106ae0 con 0x7f48001008a0 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.805+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f47f00226e0 con 0x7f48001008a0 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.805+0000 7f47fcff9700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 0x7f47ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.805+0000 7f47ff7fe700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 0x7f47ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.806+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f47f00ce350 con 0x7f48001008a0 2023-10-08T10:03:54.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.806+0000 7f47ff7fe700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 0x7f47ec0a3940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005950 tx=0x7f47e80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:54.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:54.809+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f47f00976b0 con 0x7f48001008a0 2023-10-08T10:03:54.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:54 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/529985091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.000+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f480010afd0 con 0x7f48001008a0 2023-10-08T10:03:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.001+0000 7f47fcff9700 1 -- 172.21.15.97:0/1513619347 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47f009acd0 con 0x7f48001008a0 2023-10-08T10:03:55.003 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 msgr2=0x7f47ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 0x7f47ec0a3940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005950 tx=0x7f47e80058e0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 msgr2=0x7f480019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009fd0 tx=0x7f47f0005f10 comp rx=0 tx=0).stop 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 shutdown_connections 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f47ec0a1480 0x7f47ec0a3940 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4800101240 0x7f480019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48000ff6b0 0x7f4800069810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 --2- 172.21.15.97:0/1513619347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48001008a0 0x7f480019b3b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 >> 172.21.15.97:0/1513619347 conn(0x7f48000fac40 msgr2=0x7f4800103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 shutdown_connections 2023-10-08T10:03:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.003+0000 7f4805b65700 1 -- 172.21.15.97:0/1513619347 wait complete. 2023-10-08T10:03:55.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T10:03:55.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 -- 172.21.15.97:0/2953874326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 msgr2=0x7f2ec0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/2953874326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0100cf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40099c0 tx=0x7f2eb4009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 -- 172.21.15.97:0/2953874326 shutdown_connections 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/2953874326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec0101230 0x7f2ec0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/2953874326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/2953874326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0100cf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 -- 172.21.15.97:0/2953874326 >> 172.21.15.97:0/2953874326 conn(0x7f2ec00fac10 msgr2=0x7f2ec00fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 -- 172.21.15.97:0/2953874326 shutdown_connections 2023-10-08T10:03:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.140+0000 7f2ec5d12700 1 -- 172.21.15.97:0/2953874326 wait complete. 2023-10-08T10:03:55.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.141+0000 7f2ec5d12700 1 Processor -- start 2023-10-08T10:03:55.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.141+0000 7f2ec5d12700 1 -- start start 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 0x7f2ec019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec019e210 con 0x7f2ec0100890 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec019e350 con 0x7f2ec00ff6a0 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec019e4e0 con 0x7f2ec0101230 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58732/0 (socket says 172.21.15.97:58732) 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 -- 172.21.15.97:0/1093516018 learned_addr learned my addr 172.21.15.97:0/1093516018 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebf7fe700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 -- 172.21.15.97:0/1093516018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 msgr2=0x7f2ec019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebffff700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 0x7f2ec019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 0x7f2ec019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 -- 172.21.15.97:0/1093516018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 msgr2=0x7f2ec01966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec01966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 -- 172.21.15.97:0/1093516018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40095f0 con 0x7f2ec0100890 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebf7fe700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebffff700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 0x7f2ec019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebeffd700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009f50 tx=0x7f2eb4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb401d070 con 0x7f2ec0100890 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec019e760 con 0x7f2ec0100890 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb4022470 con 0x7f2ec0100890 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400f6c0 con 0x7f2ec0100890 2023-10-08T10:03:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.142+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec019eb80 con 0x7f2ec0100890 2023-10-08T10:03:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.143+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec0106ad0 con 0x7f2ec0100890 2023-10-08T10:03:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.144+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2eb40225e0 con 0x7f2ec0100890 2023-10-08T10:03:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.145+0000 7f2ebcff9700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 0x7f2ea80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.145+0000 7f2ebf7fe700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 0x7f2ea80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.145+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2eb40ce590 con 0x7f2ec0100890 2023-10-08T10:03:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.145+0000 7f2ebf7fe700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 0x7f2ea80a3940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ec01006f0 tx=0x7f2eb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.149+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2eb4097aa0 con 0x7f2ec0100890 2023-10-08T10:03:55.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.339+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ec010c2f0 con 0x7f2ec0100890 2023-10-08T10:03:55.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.339+0000 7f2ebcff9700 1 -- 172.21.15.97:0/1093516018 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2eb40270a0 con 0x7f2ec0100890 2023-10-08T10:03:55.342 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-10-08T10:03:55.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 msgr2=0x7f2ea80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 0x7f2ea80a3940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ec01006f0 tx=0x7f2eb0005f90 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 msgr2=0x7f2ec0196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009f50 tx=0x7f2eb4003970 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 shutdown_connections 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f2ea80a1480 0x7f2ea80a3940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0101230 0x7f2ec019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ec00ff6a0 0x7f2ec01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 --2- 172.21.15.97:0/1093516018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec0100890 0x7f2ec0196c20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.341+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 >> 172.21.15.97:0/1093516018 conn(0x7f2ec00fac10 msgr2=0x7f2ec0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.342+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 shutdown_connections 2023-10-08T10:03:55.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.342+0000 7f2ec5d12700 1 -- 172.21.15.97:0/1093516018 wait complete. 2023-10-08T10:03:55.356 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T10:03:55.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 -- 172.21.15.97:0/3053496925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 msgr2=0x7f39fc0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/3053496925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc0ffac0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f39f80099c0 tx=0x7f39f8009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 -- 172.21.15.97:0/3053496925 shutdown_connections 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/3053496925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/3053496925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc0ffac0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/3053496925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 0x7f39fc100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.479+0000 7f3a03ed8700 1 -- 172.21.15.97:0/3053496925 >> 172.21.15.97:0/3053496925 conn(0x7f39fc0fac10 msgr2=0x7f39fc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- 172.21.15.97:0/3053496925 shutdown_connections 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- 172.21.15.97:0/3053496925 wait complete. 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 Processor -- start 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- start start 2023-10-08T10:03:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 0x7f39fc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e230 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e3a0 con 0x7f39fc101230 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a03ed8700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc19e560 con 0x7f39fc100890 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a01c74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a01c74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58744/0 (socket says 172.21.15.97:58744) 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.480+0000 7f3a01c74700 1 -- 172.21.15.97:0/1699358860 learned_addr learned my addr 172.21.15.97:0/1699358860 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01473700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 0x7f39fc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 -- 172.21.15.97:0/1699358860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 msgr2=0x7f39fc19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a02475700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 0x7f39fc19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 -- 172.21.15.97:0/1699358860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 msgr2=0x7f39fc19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 -- 172.21.15.97:0/1699358860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f80095f0 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a02475700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a01c74700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009f50 tx=0x7f39f8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f801d070 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f8022470 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc19e780 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800f6c0 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.481+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc19eba0 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.482+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc106b20 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.483+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f39f80225e0 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.484+0000 7f39f2ffd700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 0x7f39e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.484+0000 7f3a01473700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 0x7f39e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.484+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f39f80ceab0 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.484+0000 7f3a01473700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 0x7f39e80a3940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f39fc068910 tx=0x7f39ec005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.487+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f39f8097e70 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.678+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39fc10af00 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.679+0000 7f39f2ffd700 1 -- 172.21.15.97:0/1699358860 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f39f809b490 con 0x7f39fc0ff6a0 2023-10-08T10:03:55.681 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 msgr2=0x7f39e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 0x7f39e80a3940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f39fc068910 tx=0x7f39ec005cf0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 msgr2=0x7f39fc068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009f50 tx=0x7f39f8003970 comp rx=0 tx=0).stop 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 shutdown_connections 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f39e80a1480 0x7f39e80a3940 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39fc101230 0x7f39fc19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39fc0ff6a0 0x7f39fc068370 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 --2- 172.21.15.97:0/1699358860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39fc100890 0x7f39fc19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 >> 172.21.15.97:0/1699358860 conn(0x7f39fc0fac10 msgr2=0x7f39fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 shutdown_connections 2023-10-08T10:03:55.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.681+0000 7f3a03ed8700 1 -- 172.21.15.97:0/1699358860 wait complete. 2023-10-08T10:03:55.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:55 smithi097 ceph-mon[99491]: pgmap v563: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:55.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:55 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3855227063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:55 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1513619347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:55 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1093516018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T10:03:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:55 smithi186 ceph-mon[104393]: pgmap v563: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:55 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3855227063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:55 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1513619347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:55 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1093516018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 -- 172.21.15.97:0/3017630171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0fee20 msgr2=0x7f5f1c0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 --2- 172.21.15.97:0/3017630171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c0ff240 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000aaa0 tx=0x7f5f1000adb0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 -- 172.21.15.97:0/3017630171 shutdown_connections 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 --2- 172.21.15.97:0/3017630171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c100930 0x7f5f1c107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 --2- 172.21.15.97:0/3017630171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c0ff240 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 --2- 172.21.15.97:0/3017630171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fff70 0x7f5f1c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 -- 172.21.15.97:0/3017630171 >> 172.21.15.97:0/3017630171 conn(0x7f5f1c0fa390 msgr2=0x7f5f1c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 -- 172.21.15.97:0/3017630171 shutdown_connections 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.816+0000 7f5f210e5700 1 -- 172.21.15.97:0/3017630171 wait complete. 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 Processor -- start 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 -- start start 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c0fff70 0x7f5f1c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c19e200 con 0x7f5f1c100930 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c19e340 con 0x7f5f1c0fee20 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f210e5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c19e4d0 con 0x7f5f1c0fff70 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58760/0 (socket says 172.21.15.97:58760) 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 -- 172.21.15.97:0/2024297182 learned_addr learned my addr 172.21.15.97:0/2024297182 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1a59c700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c0fff70 0x7f5f1c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1ad9d700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 -- 172.21.15.97:0/2024297182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c0fff70 msgr2=0x7f5f1c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c0fff70 0x7f5f1c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 -- 172.21.15.97:0/2024297182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 msgr2=0x7f5f1c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1b59e700 1 -- 172.21.15.97:0/2024297182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1000a700 con 0x7f5f1c100930 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.817+0000 7f5f1ad9d700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f1b59e700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007ee0 tx=0x7f5f1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14017070 con 0x7f5f1c100930 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c19e7b0 con 0x7f5f1c100930 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f1400f040 con 0x7f5f1c100930 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14005740 con 0x7f5f1c100930 2023-10-08T10:03:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.818+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c19ecc0 con 0x7f5f1c100930 2023-10-08T10:03:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.819+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c066ca0 con 0x7f5f1c100930 2023-10-08T10:03:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.820+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5f1400dc60 con 0x7f5f1c100930 2023-10-08T10:03:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.820+0000 7f5f03fff700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.820+0000 7f5f1ad9d700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.821+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f140cc860 con 0x7f5f1c100930 2023-10-08T10:03:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.821+0000 7f5f1ad9d700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 0x7f5f040a3940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c6d0 tx=0x7f5f10003c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:55.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:55.825+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5f14095b90 con 0x7f5f1c100930 2023-10-08T10:03:55.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:55 smithi178 ceph-mon[103156]: pgmap v563: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:55.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:55 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3855227063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:55 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1513619347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T10:03:55.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:55 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1093516018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T10:03:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.032+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f1c109e10 con 0x7f5f1c100930 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.032+0000 7f5f03fff700 1 -- 172.21.15.97:0/2024297182 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+591 (secure 0 0 0) 0x7f5f140991b0 con 0x7f5f1c100930 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4256028615 2023-10-09T09:47:45.797606+0000 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/1217300114 2023-10-09T09:47:22.724740+0000 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/4047632094 2023-10-09T09:47:45.797606+0000 2023-10-08T10:03:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3827116852 2023-10-09T09:47:22.724740+0000 2023-10-08T10:03:56.035 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/1873332931 2023-10-09T09:47:45.797606+0000 2023-10-08T10:03:56.035 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/353854862 2023-10-09T09:47:22.724740+0000 2023-10-08T10:03:56.035 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/2928584164 2023-10-09T09:47:22.724740+0000 2023-10-08T10:03:56.035 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3484524433 2023-10-09T09:47:45.797606+0000 2023-10-08T10:03:56.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.034+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 msgr2=0x7f5f040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.034+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 0x7f5f040a3940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c6d0 tx=0x7f5f10003c30 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.034+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 msgr2=0x7f5f1c19dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.034+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007ee0 tx=0x7f5f1400d3b0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 shutdown_connections 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c100930 0x7f5f1c19dc30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0fee20 0x7f5f1c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 --2- 172.21.15.97:0/2024297182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f1c0fff70 0x7f5f1c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 >> 172.21.15.97:0/2024297182 conn(0x7f5f1c0fa390 msgr2=0x7f5f1c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 shutdown_connections 2023-10-08T10:03:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.035+0000 7f5f210e5700 1 -- 172.21.15.97:0/2024297182 wait complete. 2023-10-08T10:03:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:listed 10 entries 2023-10-08T10:03:56.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T10:03:56.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T10:03:56.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T10:03:56.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.115 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T10:03:56.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.150 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-10-08T10:03:56.236 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T10:03:56.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T10:03:56.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-10-08T10:03:56.381 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T10:03:56.381 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-10-08T10:03:56.381 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T10:03:56.398 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3504 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/5a643881f72a24e8b4f3fd71b2a4c349060ae46c11d773569c7e761c4f830528/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/5a643881f72a24e8b4f3fd71b2a4c349060ae46c11d773569c7e761c4f830528/diff,workdir=/var/lib/containers/storage/overlay/5a643881f72a24e8b4f3fd71b2a4c349060ae46c11d773569c7e761c4f830528/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/85af0a6bac16745811e157f7d7e0b1bf9f9ad7afbac4107cf89449bcd95433c1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/85af0a6bac16745811e157f7d7e0b1bf9f9ad7afbac4107cf89449bcd95433c1/diff,workdir=/var/lib/containers/storage/overlay/85af0a6bac16745811e157f7d7e0b1bf9f9ad7afbac4107cf89449bcd95433c1/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/5431c2514ddcb6d629b885f71b31e929e5fe700da2781bb78e7bb9485231f084/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/5431c2514ddcb6d629b885f71b31e929e5fe700da2781bb78e7bb9485231f084/diff,workdir=/var/lib/containers/storage/overlay/5431c2514ddcb6d629b885f71b31e929e5fe700da2781bb78e7bb9485231f084/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/35c687c932f052f75bff2684fa6bf5a9380352b8ac6083936958d1064924d9be/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/35c687c932f052f75bff2684fa6bf5a9380352b8ac6083936958d1064924d9be/diff,workdir=/var/lib/containers/storage/overlay/35c687c932f052f75bff2684fa6bf5a9380352b8ac6083936958d1064924d9be/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/81e7cb7de7f5be8d8cd46ab774daa5bf9d12a101f768460997d3ff8a01dac0e6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/81e7cb7de7f5be8d8cd46ab774daa5bf9d12a101f768460997d3ff8a01dac0e6/diff,workdir=/var/lib/containers/storage/overlay/81e7cb7de7f5be8d8cd46ab774daa5bf9d12a101f768460997d3ff8a01dac0e6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/502a3e9f65ec51771d7bccd809738a4bd2f4d1a48aa9b2618b1db76bd56f5532/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/502a3e9f65ec51771d7bccd809738a4bd2f4d1a48aa9b2618b1db76bd56f5532/diff,workdir=/var/lib/containers/storage/overlay/502a3e9f65ec51771d7bccd809738a4bd2f4d1a48aa9b2618b1db76bd56f5532/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6880309c8f93421e16ba220e603193ef13add9d6eba1cac93b80154ee0302fc8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/6880309c8f93421e16ba220e603193ef13add9d6eba1cac93b80154ee0302fc8/diff,workdir=/var/lib/containers/storage/overlay/6880309c8f93421e16ba220e603193ef13add9d6eba1cac93b80154ee0302fc8/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/71ddf0ec124060da9168cfec1e71ff105be210c19acb8988610f8c5ccd19a583/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/71ddf0ec124060da9168cfec1e71ff105be210c19acb8988610f8c5ccd19a583/diff,workdir=/var/lib/containers/storage/overlay/71ddf0ec124060da9168cfec1e71ff105be210c19acb8988610f8c5ccd19a583/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bc8e8daa2a3054f87b3432cc64e376c6f68827430299b955ac15e698cb04aba2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/bc8e8daa2a3054f87b3432cc64e376c6f68827430299b955ac15e698cb04aba2/diff,workdir=/var/lib/containers/storage/overlay/bc8e8daa2a3054f87b3432cc64e376c6f68827430299b955ac15e698cb04aba2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6897173f511c0308d8c287bc9a09dd8beb4835ca44c16ef9cf5f551f6901f730/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TIAXXXKFB22TVHS22RRMZOBFDG:/var/lib/containers/storage/overlay/l/YLOFD444I4HL6MRFD5JUC57N3Q,upperdir=/var/lib/containers/storage/overlay/6897173f511c0308d8c287bc9a09dd8beb4835ca44c16ef9cf5f551f6901f730/diff,workdir=/var/lib/containers/storage/overlay/6897173f511c0308d8c287bc9a09dd8beb4835ca44c16ef9cf5f551f6901f730/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-08T10:03:56.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.399 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-10-08T10:03:56.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T10:03:56.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-10-08T10:03:56.579 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-10-08T10:03:56.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:03:56.581 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-08T10:03:56.592 INFO:tasks.cephadm:Teardown begin 2023-10-08T10:03:56.592 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T10:03:56.593 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T10:03:56.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1699358860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:56.616 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:56 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2024297182' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T10:03:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 -- 172.21.15.97:0/235440853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100170 msgr2=0x7f7654107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 --2- 172.21.15.97:0/235440853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100170 0x7f7654107a10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f764c00b0a0 tx=0x7f764c00b3b0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 -- 172.21.15.97:0/235440853 shutdown_connections 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 --2- 172.21.15.97:0/235440853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100170 0x7f7654107a10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 --2- 172.21.15.97:0/235440853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100b60 0x7f7654100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 --2- 172.21.15.97:0/235440853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 -- 172.21.15.97:0/235440853 >> 172.21.15.97:0/235440853 conn(0x7f76540fa2c0 msgr2=0x7f76540fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 -- 172.21.15.97:0/235440853 shutdown_connections 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.713+0000 7f765b030700 1 -- 172.21.15.97:0/235440853 wait complete. 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 Processor -- start 2023-10-08T10:03:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 -- start start 2023-10-08T10:03:56.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:56.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100170 0x7f765419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:56.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:56.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419e240 con 0x7f7654100b60 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419e3b0 con 0x7f7654100170 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f765b030700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419e570 con 0x7f76540ff7b0 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58774/0 (socket says 172.21.15.97:58774) 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 -- 172.21.15.97:0/1071813947 learned_addr learned my addr 172.21.15.97:0/1071813947 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f7653fff700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100170 0x7f765419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f7658dcc700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:56.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 -- 172.21.15.97:0/1071813947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 msgr2=0x7f76540ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 -- 172.21.15.97:0/1071813947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100170 msgr2=0x7f765419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100170 0x7f765419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f76595cd700 1 -- 172.21.15.97:0/1071813947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009d00 con 0x7f7654100b60 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.714+0000 7f7658dcc700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:56.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f76595cd700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f764c006070 tx=0x7f764c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:56.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00e040 con 0x7f7654100b60 2023-10-08T10:03:56.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764c01db50 con 0x7f7654100b60 2023-10-08T10:03:56.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765419e790 con 0x7f7654100b60 2023-10-08T10:03:56.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c01bc00 con 0x7f7654100b60 2023-10-08T10:03:56.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.715+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765419ec10 con 0x7f7654100b60 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.716+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7654106980 con 0x7f7654100b60 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.717+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f764c01dcc0 con 0x7f7654100b60 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.717+0000 7f7651ffb700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 0x7f76440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.717+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f764c0ced70 con 0x7f7654100b60 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.717+0000 7f7658dcc700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 0x7f76440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:56.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.718+0000 7f7658dcc700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 0x7f76440a3940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7648005fd0 tx=0x7f7648005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:56.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.722+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f764c098130 con 0x7f7654100b60 2023-10-08T10:03:56.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1699358860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:56.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:56 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2024297182' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T10:03:56.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1699358860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T10:03:56.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:56 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2024297182' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T10:03:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.964+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765402cfb0 con 0x7f7654100b60 2023-10-08T10:03:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.965+0000 7f7651ffb700 1 -- 172.21.15.97:0/1071813947 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f764c09b750 con 0x7f7654100b60 2023-10-08T10:03:56.967 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:56.967 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T10:03:16.107336+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":132,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 msgr2=0x7f76440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 0x7f76440a3940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7648005fd0 tx=0x7f7648005ee0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 msgr2=0x7f765419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f764c006070 tx=0x7f764c004d00 comp rx=0 tx=0).stop 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 shutdown_connections 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f76440a1480 0x7f76440a3940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7654100b60 0x7f765419da50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76540ff7b0 0x7f76540ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 --2- 172.21.15.97:0/1071813947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7654100170 0x7f765419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 >> 172.21.15.97:0/1071813947 conn(0x7f76540fa2c0 msgr2=0x7f7654103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 shutdown_connections 2023-10-08T10:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:56.967+0000 7f765b030700 1 -- 172.21.15.97:0/1071813947 wait complete. 2023-10-08T10:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2023-10-08T10:03:56.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T10:03:57.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/1019120349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181006c0 msgr2=0x7f5e18100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/1019120349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181006c0 0x7f5e18100ae0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5e080099c0 tx=0x7f5e08009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/1019120349 shutdown_connections 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/1019120349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e18102250 0x7f5e18108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/1019120349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e181006c0 0x7f5e18100ae0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/1019120349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181018b0 0x7f5e18101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/1019120349 >> 172.21.15.97:0/1019120349 conn(0x7f5e180fbc50 msgr2=0x7f5e180fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/1019120349 shutdown_connections 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.142+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/1019120349 wait complete. 2023-10-08T10:03:57.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.143+0000 7f5e1e0a9700 1 Processor -- start 2023-10-08T10:03:57.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.143+0000 7f5e1e0a9700 1 -- start start 2023-10-08T10:03:57.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181006c0 0x7f5e18196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e181018b0 0x7f5e18196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819e160 con 0x7f5e18102250 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819e2a0 con 0x7f5e181018b0 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819e430 con 0x7f5e181006c0 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58788/0 (socket says 172.21.15.97:58788) 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 -- 172.21.15.97:0/178638862 learned_addr learned my addr 172.21.15.97:0/178638862 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e177fe700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181006c0 0x7f5e18196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e16ffd700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e181018b0 0x7f5e18196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 -- 172.21.15.97:0/178638862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181006c0 msgr2=0x7f5e18196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181006c0 0x7f5e18196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 -- 172.21.15.97:0/178638862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e181018b0 msgr2=0x7f5e18196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e181018b0 0x7f5e18196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 -- 172.21.15.97:0/178638862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080095f0 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e17fff700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000ef30 tx=0x7f5e1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10018070 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1819e6b0 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e1000f040 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.144+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000cdb0 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.145+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1819ec10 con 0x7f5e18102250 2023-10-08T10:03:57.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.145+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18107b90 con 0x7f5e18102250 2023-10-08T10:03:57.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.146+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5e10003a00 con 0x7f5e18102250 2023-10-08T10:03:57.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.147+0000 7f5e14ff9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 0x7f5e000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.147+0000 7f5e177fe700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 0x7f5e000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.147+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5e100ccca0 con 0x7f5e18102250 2023-10-08T10:03:57.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.148+0000 7f5e177fe700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 0x7f5e000a3940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e08003730 tx=0x7f5e08000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.152+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5e10096000 con 0x7f5e18102250 2023-10-08T10:03:57.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.386+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e18066bc0 con 0x7f5e18102250 2023-10-08T10:03:57.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.387+0000 7f5e14ff9700 1 -- 172.21.15.97:0/178638862 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f5e10099620 con 0x7f5e18102250 2023-10-08T10:03:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:57.389 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2090847706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2090847706},{"type":"v1","addr":"172.21.15.97:6835","nonce":2090847706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3214411301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3214411301},{"type":"v1","addr":"172.21.15.97:6837","nonce":3214411301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14754,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1211581786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1211581786},{"type":"v1","addr":"172.21.15.97:6839","nonce":1211581786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/3143185366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":3143185366},{"type":"v1","addr":"172.21.15.97:6841","nonce":3143185366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24500,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1070195466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1070195466},{"type":"v1","addr":"172.21.15.178:6833","nonce":1070195466}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3404847957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3404847957},{"type":"v1","addr":"172.21.15.186:6833","nonce":3404847957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:53:56.599415+0000","modified":"2023-10-08T10:03:16.107336+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24562,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/2717102390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":2717102390},{"type":"v1","addr":"172.21.15.178:6835","nonce":2717102390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/3587378302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":3587378302},{"type":"v1","addr":"172.21.15.186:6835","nonce":3587378302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/2015632800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2015632800},{"type":"v1","addr":"172.21.15.178:6837","nonce":2015632800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.186:6837/4190406762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":4190406762},{"type":"v1","addr":"172.21.15.186:6837","nonce":4190406762}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.186:6839/2120449823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2120449823},{"type":"v1","addr":"172.21.15.186:6839","nonce":2120449823}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":132,"addr":"172.21.15.178:6839/1665352653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":1665352653},{"type":"v1","addr":"172.21.15.178:6839","nonce":1665352653}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 msgr2=0x7f5e000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 0x7f5e000a3940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e08003730 tx=0x7f5e08000bc0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 msgr2=0x7f5e1819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000ef30 tx=0x7f5e1000c3a0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 shutdown_connections 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f5e000a1480 0x7f5e000a3940 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e18102250 0x7f5e1819db90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e181006c0 0x7f5e18196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 --2- 172.21.15.97:0/178638862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e181018b0 0x7f5e18196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 >> 172.21.15.97:0/178638862 conn(0x7f5e180fbc50 msgr2=0x7f5e18104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:57.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 shutdown_connections 2023-10-08T10:03:57.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.390+0000 7f5e1e0a9700 1 -- 172.21.15.97:0/178638862 wait complete. 2023-10-08T10:03:57.393 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 52 2023-10-08T10:03:57.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.523+0000 7f4eb0431700 1 -- 172.21.15.97:0/1451554128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 msgr2=0x7f4ea8100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.523+0000 7f4eb0431700 1 --2- 172.21.15.97:0/1451554128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8100bf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40099c0 tx=0x7f4ea4009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- 172.21.15.97:0/1451554128 shutdown_connections 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- 172.21.15.97:0/1451554128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea81079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- 172.21.15.97:0/1451554128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- 172.21.15.97:0/1451554128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8100bf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- 172.21.15.97:0/1451554128 >> 172.21.15.97:0/1451554128 conn(0x7f4ea80fab10 msgr2=0x7f4ea80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- 172.21.15.97:0/1451554128 shutdown_connections 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- 172.21.15.97:0/1451554128 wait complete. 2023-10-08T10:03:57.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 Processor -- start 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- start start 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819e170 con 0x7f4ea8100790 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819e2b0 con 0x7f4ea80ff5a0 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.524+0000 7f4eb0431700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819e440 con 0x7f4ea8101130 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58798/0 (socket says 172.21.15.97:58798) 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 -- 172.21.15.97:0/2656202500 learned_addr learned my addr 172.21.15.97:0/2656202500 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eae9ce700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eae1cd700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 -- 172.21.15.97:0/2656202500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 msgr2=0x7f4ea819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 -- 172.21.15.97:0/2656202500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 msgr2=0x7f4ea8196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea8196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 -- 172.21.15.97:0/2656202500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40095f0 con 0x7f4ea8100790 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eae1cd700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eae9ce700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4ead9cc700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009f50 tx=0x7f4ea4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea401d070 con 0x7f4ea8100790 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea819e6c0 con 0x7f4ea8100790 2023-10-08T10:03:57.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ea4022470 con 0x7f4ea8100790 2023-10-08T10:03:57.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea400f6c0 con 0x7f4ea8100790 2023-10-08T10:03:57.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.525+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea819eb40 con 0x7f4ea8100790 2023-10-08T10:03:57.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.526+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea8067450 con 0x7f4ea8100790 2023-10-08T10:03:57.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.527+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4ea40225e0 con 0x7f4ea8100790 2023-10-08T10:03:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.529+0000 7f4e9f7fe700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 0x7f4e940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.529+0000 7f4eae1cd700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 0x7f4e940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.529+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ea40cec70 con 0x7f4ea8100790 2023-10-08T10:03:57.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.529+0000 7f4eae1cd700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 0x7f4e940a3940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4e98005fd0 tx=0x7f4e98005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.532+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ea40980c0 con 0x7f4ea8100790 2023-10-08T10:03:57.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:57 smithi097 ceph-mon[99491]: pgmap v564: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1071813947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:57.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:57 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/178638862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.725+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea810af80 con 0x7f4ea8100790 2023-10-08T10:03:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.727+0000 7f4e9f7fe700 1 -- 172.21.15.97:0/2656202500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24545 (secure 0 0 0) 0x7f4ea409b6e0 con 0x7f4ea8100790 2023-10-08T10:03:57.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-10-08T10:03:57.729 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"7ec07914-65bf-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:46:49.063848+0000","modified":"2023-10-08T09:55:47.251150+0000","last_up_change":"2023-10-08T09:53:16.084078+0000","last_in_change":"2023-10-08T09:52:59.357480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:50:14.128447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:53:54.548005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:53:55.793692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:53:59.455547+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"31b0d724-2bfc-4c58-8ac5-38962f5d300a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6803","nonce":3104991007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6805","nonce":3104991007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6809","nonce":3104991007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3104991007},{"type":"v1","addr":"172.21.15.97:6807","nonce":3104991007}]},"public_addr":"172.21.15.97:6803/3104991007","cluster_addr":"172.21.15.97:6805/3104991007","heartbeat_back_addr":"172.21.15.97:6809/3104991007","heartbeat_front_addr":"172.21.15.97:6807/3104991007","state":["exists","up"]},{"osd":1,"uuid":"946de8df-41b4-46e5-87da-4822fa096d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6801","nonce":4284206053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6803","nonce":4284206053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6807","nonce":4284206053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4284206053},{"type":"v1","addr":"172.21.15.178:6805","nonce":4284206053}]},"public_addr":"172.21.15.178:6801/4284206053","cluster_addr":"172.21.15.178:6803/4284206053","heartbeat_back_addr":"172.21.15.178:6807/4284206053","heartbeat_front_addr":"172.21.15.178:6805/4284206053","state":["exists","up"]},{"osd":2,"uuid":"7dcdc682-9882-47fb-889a-4d412fbc7433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6801","nonce":1978590862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6803","nonce":1978590862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6807","nonce":1978590862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1978590862},{"type":"v1","addr":"172.21.15.186:6805","nonce":1978590862}]},"public_addr":"172.21.15.186:6801/1978590862","cluster_addr":"172.21.15.186:6803/1978590862","heartbeat_back_addr":"172.21.15.186:6807/1978590862","heartbeat_front_addr":"172.21.15.186:6805/1978590862","state":["exists","up"]},{"osd":3,"uuid":"1a30f31c-10a5-4538-8fd9-35acf14be3eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6811","nonce":105487765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6813","nonce":105487765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6817","nonce":105487765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":105487765},{"type":"v1","addr":"172.21.15.97:6815","nonce":105487765}]},"public_addr":"172.21.15.97:6811/105487765","cluster_addr":"172.21.15.97:6813/105487765","heartbeat_back_addr":"172.21.15.97:6817/105487765","heartbeat_front_addr":"172.21.15.97:6815/105487765","state":["exists","up"]},{"osd":4,"uuid":"bd3cbf7e-d161-4d7f-959e-5943a5a63bd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6809","nonce":1866336071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6811","nonce":1866336071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6815","nonce":1866336071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1866336071},{"type":"v1","addr":"172.21.15.178:6813","nonce":1866336071}]},"public_addr":"172.21.15.178:6809/1866336071","cluster_addr":"172.21.15.178:6811/1866336071","heartbeat_back_addr":"172.21.15.178:6815/1866336071","heartbeat_front_addr":"172.21.15.178:6813/1866336071","state":["exists","up"]},{"osd":5,"uuid":"7091360b-fa15-4e88-a5cc-44dcebef03c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6809","nonce":2325993669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6811","nonce":2325993669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6815","nonce":2325993669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2325993669},{"type":"v1","addr":"172.21.15.186:6813","nonce":2325993669}]},"public_addr":"172.21.15.186:6809/2325993669","cluster_addr":"172.21.15.186:6811/2325993669","heartbeat_back_addr":"172.21.15.186:6815/2325993669","heartbeat_front_addr":"172.21.15.186:6813/2325993669","state":["exists","up"]},{"osd":6,"uuid":"facc86e5-e587-42d9-bbae-6f98916a084d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6819","nonce":3267640836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6821","nonce":3267640836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6825","nonce":3267640836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3267640836},{"type":"v1","addr":"172.21.15.97:6823","nonce":3267640836}]},"public_addr":"172.21.15.97:6819/3267640836","cluster_addr":"172.21.15.97:6821/3267640836","heartbeat_back_addr":"172.21.15.97:6825/3267640836","heartbeat_front_addr":"172.21.15.97:6823/3267640836","state":["exists","up"]},{"osd":7,"uuid":"9764524a-05a3-4d06-83a8-c9c4ba6ec38c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6817","nonce":2550553059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6819","nonce":2550553059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6823","nonce":2550553059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2550553059},{"type":"v1","addr":"172.21.15.178:6821","nonce":2550553059}]},"public_addr":"172.21.15.178:6817/2550553059","cluster_addr":"172.21.15.178:6819/2550553059","heartbeat_back_addr":"172.21.15.178:6823/2550553059","heartbeat_front_addr":"172.21.15.178:6821/2550553059","state":["exists","up"]},{"osd":8,"uuid":"f4415829-318a-4bb8-8e77-feda54434e5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6817","nonce":426737999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6819","nonce":426737999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6823","nonce":426737999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":426737999},{"type":"v1","addr":"172.21.15.186:6821","nonce":426737999}]},"public_addr":"172.21.15.186:6817/426737999","cluster_addr":"172.21.15.186:6819/426737999","heartbeat_back_addr":"172.21.15.186:6823/426737999","heartbeat_front_addr":"172.21.15.186:6821/426737999","state":["exists","up"]},{"osd":9,"uuid":"0c92bbd2-e231-4a8e-ab2c-cd63e4aaef91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6827","nonce":3522179908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6829","nonce":3522179908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6833","nonce":3522179908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3522179908},{"type":"v1","addr":"172.21.15.97:6831","nonce":3522179908}]},"public_addr":"172.21.15.97:6827/3522179908","cluster_addr":"172.21.15.97:6829/3522179908","heartbeat_back_addr":"172.21.15.97:6833/3522179908","heartbeat_front_addr":"172.21.15.97:6831/3522179908","state":["exists","up"]},{"osd":10,"uuid":"9b0e6cc7-619d-466b-9ede-4bfc9d404bba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6825","nonce":3571291491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6827","nonce":3571291491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6831","nonce":3571291491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3571291491},{"type":"v1","addr":"172.21.15.178:6829","nonce":3571291491}]},"public_addr":"172.21.15.178:6825/3571291491","cluster_addr":"172.21.15.178:6827/3571291491","heartbeat_back_addr":"172.21.15.178:6831/3571291491","heartbeat_front_addr":"172.21.15.178:6829/3571291491","state":["exists","up"]},{"osd":11,"uuid":"34036837-0a5c-4d09-b8cb-ff074305f8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6825","nonce":1987590926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6827","nonce":1987590926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6831","nonce":1987590926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":1987590926},{"type":"v1","addr":"172.21.15.186:6829","nonce":1987590926}]},"public_addr":"172.21.15.186:6825/1987590926","cluster_addr":"172.21.15.186:6827/1987590926","heartbeat_back_addr":"172.21.15.186:6831/1987590926","heartbeat_front_addr":"172.21.15.186:6829/1987590926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:29.645779+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:49:50.344550+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:11.555067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:31.959947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:50:53.171214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:13.049288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:33.428734+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:51:54.224692+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:14.009612+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:34.327878+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:52:54.838877+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:53:14.687494+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4256028615":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6801/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6800/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/1217300114":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/4047632094":"2023-10-09T09:47:45.797606+0000","172.21.15.97:6800/3827116852":"2023-10-09T09:47:22.724740+0000","172.21.15.97:6801/1873332931":"2023-10-09T09:47:45.797606+0000","172.21.15.97:0/353854862":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/2928584164":"2023-10-09T09:47:22.724740+0000","172.21.15.97:0/3484524433":"2023-10-09T09:47:45.797606+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T10:03:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.728+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 msgr2=0x7f4e940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.728+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 0x7f4e940a3940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4e98005fd0 tx=0x7f4e98005dc0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.728+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 msgr2=0x7f4ea8196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.728+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009f50 tx=0x7f4ea4003970 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 shutdown_connections 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f4e940a1480 0x7f4e940a3940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ea8101130 0x7f4ea819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea80ff5a0 0x7f4ea8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 --2- 172.21.15.97:0/2656202500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100790 0x7f4ea8196b80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 >> 172.21.15.97:0/2656202500 conn(0x7f4ea80fab10 msgr2=0x7f4ea8067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 shutdown_connections 2023-10-08T10:03:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.729+0000 7f4eb0431700 1 -- 172.21.15.97:0/2656202500 wait complete. 2023-10-08T10:03:57.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T10:03:57.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:57 smithi186 ceph-mon[104393]: pgmap v564: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:57.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1071813947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:57.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:57 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/178638862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/1830989179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 msgr2=0x7ff0d8100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/1830989179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8100cf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80099c0 tx=0x7ff0c8009cd0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/1830989179 shutdown_connections 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/1830989179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 0x7ff0d8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/1830989179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 0x7ff0d80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/1830989179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8100cf0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/1830989179 >> 172.21.15.97:0/1830989179 conn(0x7ff0d80fac10 msgr2=0x7ff0d80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/1830989179 shutdown_connections 2023-10-08T10:03:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/1830989179 wait complete. 2023-10-08T10:03:57.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.865+0000 7ff0dd4e0700 1 Processor -- start 2023-10-08T10:03:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 -- start start 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 0x7ff0d8195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 0x7ff0d819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819d0b0 con 0x7ff0d8100890 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819d1f0 con 0x7ff0d8101230 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819d380 con 0x7ff0d80ff6a0 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58812/0 (socket says 172.21.15.97:58812) 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 -- 172.21.15.97:0/48755886 learned_addr learned my addr 172.21.15.97:0/48755886 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:57.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d6ffd700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 0x7ff0d8195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d77fe700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 0x7ff0d819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 -- 172.21.15.97:0/48755886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 msgr2=0x7ff0d8195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 0x7ff0d8195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 -- 172.21.15.97:0/48755886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 msgr2=0x7ff0d819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 0x7ff0d819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 -- 172.21.15.97:0/48755886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80095f0 con 0x7ff0d8100890 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0d67fc700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80036f0 tx=0x7ff0c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801d070 con 0x7ff0d8100890 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d819d600 con 0x7ff0d8100890 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c8004090 con 0x7ff0d8100890 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.866+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8022d40 con 0x7ff0d8100890 2023-10-08T10:03:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.867+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d819dab0 con 0x7ff0d8100890 2023-10-08T10:03:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.867+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d8106a50 con 0x7ff0d8100890 2023-10-08T10:03:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.869+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff0c8022470 con 0x7ff0d8100890 2023-10-08T10:03:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.869+0000 7ff0bffff700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 0x7ff0c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:57.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.869+0000 7ff0d6ffd700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 0x7ff0c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.869+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff0c80ce5b0 con 0x7ff0d8100890 2023-10-08T10:03:57.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.869+0000 7ff0d6ffd700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 0x7ff0c00a3940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0d81006f0 tx=0x7ff0cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:57.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:57.872+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0c8097910 con 0x7ff0d8100890 2023-10-08T10:03:57.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:57 smithi178 ceph-mon[103156]: pgmap v564: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-08T10:03:57.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1071813947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:57.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:57 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/178638862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T10:03:58.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.109+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff0d8067530 con 0x7ff0d8100890 2023-10-08T10:03:58.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.112+0000 7ff0bffff700 1 -- 172.21.15.97:0/48755886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0c809af30 con 0x7ff0d8100890 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.114+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 msgr2=0x7ff0c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.114+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 0x7ff0c00a3940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0d81006f0 tx=0x7ff0cc00b410 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 msgr2=0x7ff0d8195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80036f0 tx=0x7ff0c80047b0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 shutdown_connections 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7ff0c00a1480 0x7ff0c00a3940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d8101230 0x7ff0d819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0d80ff6a0 0x7ff0d8195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 --2- 172.21.15.97:0/48755886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d8100890 0x7ff0d8195ac0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 >> 172.21.15.97:0/48755886 conn(0x7ff0d80fac10 msgr2=0x7ff0d8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:58.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 shutdown_connections 2023-10-08T10:03:58.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.115+0000 7ff0dd4e0700 1 -- 172.21.15.97:0/48755886 wait complete. 2023-10-08T10:03:58.118 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T10:03:58.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T10:03:58.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- 172.21.15.97:0/1473138617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe800102350 msgr2=0x7fe800108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- 172.21.15.97:0/1473138617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe800102350 0x7fe800108c00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800b0d0 tx=0x7fe7f800b3e0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- 172.21.15.97:0/1473138617 shutdown_connections 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- 172.21.15.97:0/1473138617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe800102350 0x7fe800108c00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- 172.21.15.97:0/1473138617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- 172.21.15.97:0/1473138617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8001019b0 0x7fe800101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- 172.21.15.97:0/1473138617 >> 172.21.15.97:0/1473138617 conn(0x7fe8000fbd30 msgr2=0x7fe8000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- 172.21.15.97:0/1473138617 shutdown_connections 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- 172.21.15.97:0/1473138617 wait complete. 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 Processor -- start 2023-10-08T10:03:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- start start 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 0x7fe80019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80019e2b0 con 0x7fe8001019b0 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80019e3f0 con 0x7fe8001007c0 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe807780700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80019e580 con 0x7fe800102350 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe804d1b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe804d1b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58826/0 (socket says 172.21.15.97:58826) 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe804d1b700 1 -- 172.21.15.97:0/149066914 learned_addr learned my addr 172.21.15.97:0/149066914 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.255+0000 7fe80551c700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe805d1d700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 0x7fe80019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 -- 172.21.15.97:0/149066914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 msgr2=0x7fe80019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 0x7fe80019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 -- 172.21.15.97:0/149066914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 msgr2=0x7fe800196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 -- 172.21.15.97:0/149066914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8009d00 con 0x7fe8001019b0 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe80551c700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe805d1d700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 0x7fe80019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe804d1b700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009b40 con 0x7fe8001019b0 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7fc009ca0 con 0x7fe8001019b0 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80019e860 con 0x7fe8001019b0 2023-10-08T10:03:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00f620 con 0x7fe8001019b0 2023-10-08T10:03:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.256+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80019ed70 con 0x7fe8001019b0 2023-10-08T10:03:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.258+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe800107b10 con 0x7fe8001019b0 2023-10-08T10:03:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.259+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe7fc010460 con 0x7fe8001019b0 2023-10-08T10:03:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.259+0000 7fe7f67fc700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 0x7fe7ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.259+0000 7fe80551c700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 0x7fe7ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.259+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fe7fc0cd4f0 con 0x7fe8001019b0 2023-10-08T10:03:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.261+0000 7fe80551c700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 0x7fe7ec0a3940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005fd0 tx=0x7fe7f0005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:58.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.264+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7fc096720 con 0x7fe8001019b0 2023-10-08T10:03:58.452 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a[99467]: 2023-10-08T10:03:58.110+0000 7f22ff52e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T10:03:58.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mds[140362]: -- 172.21.15.178:0/734262257 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ed42a51e0 con 0x555ed427b000 2023-10-08T10:03:58.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: starting mds.b at 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mds[139684]: -- 172.21.15.178:0/219238954 <== mon.2 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563a189471e0 con 0x563a1891cc00 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: starting mds.e at 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mds[140022]: -- 172.21.15.178:0/4145022540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5597720a31e0 con 0x559772078c00 2023-10-08T10:03:58.456 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: starting mds.h at 2023-10-08T10:03:58.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:58.500+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe80010c190 con 0x7fe8001019b0 2023-10-08T10:03:58.517 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.517 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.517 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mds[142469]: -- 172.21.15.186:0/2825404815 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5649761351e0 con 0x56497610b000 2023-10-08T10:03:58.517 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: starting mds.c at 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mds[142127]: -- 172.21.15.186:0/1527725447 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55766b9151e0 con 0x55766b8ea800 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: starting mds.f at 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: ignoring --setuser ceph since I am not root 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: ignoring --setgroup ceph since I am not root 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mds[142810]: -- 172.21.15.186:0/3657087135 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bd03b1f1e0 con 0x55bd03af4c00 2023-10-08T10:03:58.518 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: starting mds.i at 2023-10-08T10:03:58.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/2656202500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:58.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T10:03:58.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T10:03:58.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T10:03:58.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T10:03:58.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T10:03:58.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-08T10:03:58.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:58 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/2656202500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-08T10:03:58.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:58 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/2656202500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/48755886' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T10:03:58.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T10:03:58.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-08T10:03:58.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:58 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T10:03:59.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.118+0000 7fe7f67fc700 1 -- 172.21.15.97:0/149066914 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7fc099d40 con 0x7fe8001019b0 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 msgr2=0x7fe7ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 0x7fe7ec0a3940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005fd0 tx=0x7fe7f0005e40 comp rx=0 tx=0).stop 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 msgr2=0x7fe800196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00dd10 comp rx=0 tx=0).stop 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 shutdown_connections 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fe7ec0a1480 0x7fe7ec0a3940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe800102350 0x7fe80019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8001007c0 0x7fe800196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 --2- 172.21.15.97:0/149066914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8001019b0 0x7fe800196cc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 >> 172.21.15.97:0/149066914 conn(0x7fe8000fbd30 msgr2=0x7fe800104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 shutdown_connections 2023-10-08T10:03:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.121+0000 7fe807780700 1 -- 172.21.15.97:0/149066914 wait complete. 2023-10-08T10:03:59.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T10:03:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 -- 172.21.15.97:0/2777419886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff68101230 msgr2=0x7eff68107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 --2- 172.21.15.97:0/2777419886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff68101230 0x7eff68107ae0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b0d0 tx=0x7eff6000b3e0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 -- 172.21.15.97:0/2777419886 shutdown_connections 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 --2- 172.21.15.97:0/2777419886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff68101230 0x7eff68107ae0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 --2- 172.21.15.97:0/2777419886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff680ff6a0 0x7eff680ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 --2- 172.21.15.97:0/2777419886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff68100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 -- 172.21.15.97:0/2777419886 >> 172.21.15.97:0/2777419886 conn(0x7eff680fac10 msgr2=0x7eff680fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.267+0000 7eff701bf700 1 -- 172.21.15.97:0/2777419886 shutdown_connections 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 -- 172.21.15.97:0/2777419886 wait complete. 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 Processor -- start 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 -- start start 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff6819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff68101230 0x7eff6819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6819e280 con 0x7eff680ff6a0 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6819e3f0 con 0x7eff68101230 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff701bf700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6819e5b0 con 0x7eff68100890 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff6df5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff6df5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58838/0 (socket says 172.21.15.97:58838) 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff6df5b700 1 -- 172.21.15.97:0/1117258251 learned_addr learned my addr 172.21.15.97:0/1117258251 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.268+0000 7eff6e75c700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff68101230 0x7eff6819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6d75a700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff6819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 -- 172.21.15.97:0/1117258251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 msgr2=0x7eff6819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:59.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff6819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 -- 172.21.15.97:0/1117258251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff68101230 msgr2=0x7eff6819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff68101230 0x7eff6819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 -- 172.21.15.97:0/1117258251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009d00 con 0x7eff680ff6a0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6d75a700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff6819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff6df5b700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c990 tx=0x7eff6400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64007b00 con 0x7eff680ff6a0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff64007c60 con 0x7eff680ff6a0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6819e890 con 0x7eff680ff6a0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64018660 con 0x7eff680ff6a0 2023-10-08T10:03:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.269+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6819eda0 con 0x7eff680ff6a0 2023-10-08T10:03:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.270+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff681069f0 con 0x7eff680ff6a0 2023-10-08T10:03:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.271+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7eff6400f450 con 0x7eff680ff6a0 2023-10-08T10:03:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.271+0000 7eff5effd700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 0x7eff540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:03:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.272+0000 7eff6d75a700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 0x7eff540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:03:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.272+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9307+0+0 (secure 0 0 0) 0x7eff640cd090 con 0x7eff680ff6a0 2023-10-08T10:03:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.272+0000 7eff6d75a700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 0x7eff540a3940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff58005fd0 tx=0x7eff58005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:03:59.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.276+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7eff640962c0 con 0x7eff680ff6a0 2023-10-08T10:03:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:03:59.471+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff68067530 con 0x7eff680ff6a0 2023-10-08T10:03:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: pgmap v566: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T10:03:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T10:03:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T10:03:59.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: Cluster is now healthy 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6837/601802228,v1:172.21.15.186:6839/601802228] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6834/1391029791,v1:172.21.15.186:6835/1391029791] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.186:6836/1778029591,v1:172.21.15.186:6838/1778029591] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6834/852491577,v1:172.21.15.178:6835/852491577] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6836/1642667472,v1:172.21.15.178:6837/1642667472] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: mds.? [v2:172.21.15.178:6838/3383487271,v1:172.21.15.178:6839/3383487271] up:boot 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: fsmap 12 up:standby 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T10:03:59.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T10:03:59.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T10:03:59.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T10:03:59.777 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:03:59 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: pgmap v566: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: Cluster is now healthy 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6837/601802228,v1:172.21.15.186:6839/601802228] up:boot 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6834/1391029791,v1:172.21.15.186:6835/1391029791] up:boot 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.186:6836/1778029591,v1:172.21.15.186:6838/1778029591] up:boot 2023-10-08T10:03:59.953 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6834/852491577,v1:172.21.15.178:6835/852491577] up:boot 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6836/1642667472,v1:172.21.15.178:6837/1642667472] up:boot 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: mds.? [v2:172.21.15.178:6838/3383487271,v1:172.21.15.178:6839/3383487271] up:boot 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: fsmap 12 up:standby 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T10:03:59.954 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:03:59 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:03:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: pgmap v566: 129 pgs: 129 active+clean; 9.6 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-08T10:03:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T10:03:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T10:03:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: Cluster is now healthy 2023-10-08T10:03:59.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/149066914' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6837/601802228,v1:172.21.15.186:6839/601802228] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6834/1391029791,v1:172.21.15.186:6835/1391029791] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.186:6836/1778029591,v1:172.21.15.186:6838/1778029591] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6834/852491577,v1:172.21.15.178:6835/852491577] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6836/1642667472,v1:172.21.15.178:6837/1642667472] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: mds.? [v2:172.21.15.178:6838/3383487271,v1:172.21.15.178:6839/3383487271] up:boot 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: fsmap 12 up:standby 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T10:03:59.956 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='mgr.14152 172.21.15.97:0/3121042829' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T10:03:59.957 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:03:59 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.120+0000 7eff5effd700 1 -- 172.21.15.97:0/1117258251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7eff640998e0 con 0x7eff680ff6a0 2023-10-08T10:04:00.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 msgr2=0x7eff540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:00.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 0x7eff540a3940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff58005fd0 tx=0x7eff58005dc0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 msgr2=0x7eff68068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:00.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c990 tx=0x7eff6400cca0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 shutdown_connections 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7eff540a1480 0x7eff540a3940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff68101230 0x7eff6819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff680ff6a0 0x7eff68068350 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 --2- 172.21.15.97:0/1117258251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff68100890 0x7eff6819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.124+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 >> 172.21.15.97:0/1117258251 conn(0x7eff680fac10 msgr2=0x7eff68103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.125+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 shutdown_connections 2023-10-08T10:04:00.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.125+0000 7eff701bf700 1 -- 172.21.15.97:0/1117258251 wait complete. 2023-10-08T10:04:00.127 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-10-08T10:04:00.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T10:04:00.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 -- 172.21.15.97:0/3228494426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 msgr2=0x7f0a8c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3228494426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c107ae0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400b0d0 tx=0x7f0a8400b3e0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 -- 172.21.15.97:0/3228494426 shutdown_connections 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3228494426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c107ae0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3228494426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c0ff6a0 0x7f0a8c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3228494426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c100890 0x7f0a8c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.281+0000 7f0a90c03700 1 -- 172.21.15.97:0/3228494426 >> 172.21.15.97:0/3228494426 conn(0x7f0a8c0fac10 msgr2=0x7f0a8c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- 172.21.15.97:0/3228494426 shutdown_connections 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- 172.21.15.97:0/3228494426 wait complete. 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 Processor -- start 2023-10-08T10:04:00.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- start start 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c0ff6a0 0x7f0a8c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c100890 0x7f0a8c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c102a20 con 0x7f0a8c101230 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c102b90 con 0x7f0a8c0ff6a0 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.282+0000 7f0a90c03700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a0920 con 0x7f0a8c100890 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58856/0 (socket says 172.21.15.97:58856) 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 -- 172.21.15.97:0/3593516681 learned_addr learned my addr 172.21.15.97:0/3593516681 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8a59c700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c0ff6a0 0x7f0a8c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:00.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a89d9b700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c100890 0x7f0a8c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 -- 172.21.15.97:0/3593516681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c100890 msgr2=0x7f0a8c101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c100890 0x7f0a8c101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 -- 172.21.15.97:0/3593516681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c0ff6a0 msgr2=0x7f0a8c1036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c0ff6a0 0x7f0a8c1036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 -- 172.21.15.97:0/3593516681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009d00 con 0x7f0a8c101230 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a8ad9d700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007bd0 tx=0x7f0a84009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400e040 con 0x7f0a8c101230 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a84004e60 con 0x7f0a8c101230 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c1a0b50 con 0x7f0a8c101230 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840055c0 con 0x7f0a8c101230 2023-10-08T10:04:00.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.283+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1a0f70 con 0x7f0a8c101230 2023-10-08T10:04:00.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.284+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c067530 con 0x7f0a8c101230 2023-10-08T10:04:00.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.285+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0a84012cb0 con 0x7f0a8c101230 2023-10-08T10:04:00.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.286+0000 7f0a7b7fe700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 0x7f0a740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:00.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.286+0000 7f0a8a59c700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 0x7f0a740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:00.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.286+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8753+0+0 (secure 0 0 0) 0x7f0a840ceea0 con 0x7f0a8c101230 2023-10-08T10:04:00.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.286+0000 7f0a8a59c700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 0x7f0a740a3940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c006fd0 tx=0x7f0a7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:04:00.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.290+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a84098380 con 0x7f0a8c101230 2023-10-08T10:04:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:00.486+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a8c10af60 con 0x7f0a8c101230 2023-10-08T10:04:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.441+0000 7f0a7b7fe700 1 -- 172.21.15.97:0/3593516681 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0a8409b9a0 con 0x7f0a8c101230 2023-10-08T10:04:01.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 msgr2=0x7f0a740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:01.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 0x7f0a740a3940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c006fd0 tx=0x7f0a7c008040 comp rx=0 tx=0).stop 2023-10-08T10:04:01.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 msgr2=0x7f0a8c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007bd0 tx=0x7f0a84009570 comp rx=0 tx=0).stop 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 shutdown_connections 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7f0a740a1480 0x7f0a740a3940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a8c101230 0x7f0a8c102280 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a8c0ff6a0 0x7f0a8c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 --2- 172.21.15.97:0/3593516681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a8c100890 0x7f0a8c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 >> 172.21.15.97:0/3593516681 conn(0x7f0a8c0fac10 msgr2=0x7f0a8c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 shutdown_connections 2023-10-08T10:04:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.444+0000 7f0a90c03700 1 -- 172.21.15.97:0/3593516681 wait complete. 2023-10-08T10:04:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-10-08T10:04:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:01 smithi097 ceph-mon[99491]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T10:04:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:01 smithi097 ceph-mon[99491]: pgmap v568: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:01.453 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:01 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:01 smithi178 ceph-mon[103156]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T10:04:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:01 smithi178 ceph-mon[103156]: pgmap v568: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:01.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:01 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:01.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-08T10:04:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/1117258251' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:01 smithi186 ceph-mon[104393]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T10:04:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:01 smithi186 ceph-mon[104393]: pgmap v568: 97 pgs: 97 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:01.525 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:01 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 -- 172.21.15.97:0/507016676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01019b0 msgr2=0x7fd8f0101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/507016676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01019b0 0x7fd8f0101e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00099c0 tx=0x7fd8e0009cd0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 -- 172.21.15.97:0/507016676 shutdown_connections 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/507016676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 0x7fd8f0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/507016676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01007c0 0x7fd8f0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/507016676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01019b0 0x7fd8f0101e10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 -- 172.21.15.97:0/507016676 >> 172.21.15.97:0/507016676 conn(0x7fd8f00fbd30 msgr2=0x7fd8f00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 -- 172.21.15.97:0/507016676 shutdown_connections 2023-10-08T10:04:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.607+0000 7fd8f4d31700 1 -- 172.21.15.97:0/507016676 wait complete. 2023-10-08T10:04:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 Processor -- start 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 -- start start 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01019b0 0x7fd8f0196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 0x7fd8f019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019e360 con 0x7fd8f01007c0 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019e4a0 con 0x7fd8f0102350 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8f4d31700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019e630 con 0x7fd8f01019b0 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8ee59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.608+0000 7fd8ee59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58866/0 (socket says 172.21.15.97:58866) 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 -- 172.21.15.97:0/105673727 learned_addr learned my addr 172.21.15.97:0/105673727 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-10-08T10:04:01.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8edd9b700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01019b0 0x7fd8f0196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8eed9d700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 0x7fd8f019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 -- 172.21.15.97:0/105673727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01019b0 msgr2=0x7fd8f0196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01019b0 0x7fd8f0196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 -- 172.21.15.97:0/105673727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 msgr2=0x7fd8f019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 0x7fd8f019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 -- 172.21.15.97:0/105673727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00095f0 con 0x7fd8f01007c0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8ee59c700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b500 tx=0x7fd8e400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4011830 con 0x7fd8f01007c0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f019e910 con 0x7fd8f01007c0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e4011e50 con 0x7fd8f01007c0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010dc0 con 0x7fd8f01007c0 2023-10-08T10:04:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.609+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f019ee20 con 0x7fd8f01007c0 2023-10-08T10:04:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.610+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f0107c40 con 0x7fd8f01007c0 2023-10-08T10:04:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.611+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd8e4010380 con 0x7fd8f01007c0 2023-10-08T10:04:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.612+0000 7fd8df7fe700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T10:04:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.612+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8252+0+0 (secure 0 0 0) 0x7fd8e40cccc0 con 0x7fd8f01007c0 2023-10-08T10:04:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.612+0000 7fd8edd9b700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T10:04:01.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.612+0000 7fd8edd9b700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 0x7fd8d80a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0003730 tx=0x7fd8e000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T10:04:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.619+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd8e4096310 con 0x7fd8f01007c0 2023-10-08T10:04:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:01.830+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8f0067980 con 0x7fd8f01007c0 2023-10-08T10:04:02.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.448+0000 7fd8df7fe700 1 -- 172.21.15.97:0/105673727 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd8e4099930 con 0x7fd8f01007c0 2023-10-08T10:04:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 msgr2=0x7fd8d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 0x7fd8d80a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0003730 tx=0x7fd8e000b540 comp rx=0 tx=0).stop 2023-10-08T10:04:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 msgr2=0x7fd8f0196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T10:04:02.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b500 tx=0x7fd8e400b810 comp rx=0 tx=0).stop 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 shutdown_connections 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:6800/3991091613,v1:172.21.15.97:6801/3991091613] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f0102350 0x7fd8f019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8f01007c0 0x7fd8f0196830 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 --2- 172.21.15.97:0/105673727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8f01019b0 0x7fd8f0196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 >> 172.21.15.97:0/105673727 conn(0x7fd8f00fbd30 msgr2=0x7fd8f0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 shutdown_connections 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-10-08T10:04:02.451+0000 7fd8f4d31700 1 -- 172.21.15.97:0/105673727 wait complete. 2023-10-08T10:04:02.453 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' removed 2023-10-08T10:04:02.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:04:02.510 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:04:02.557 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:04:02.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:02.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:02 smithi097 ceph-mon[99491]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T10:04:02.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:02 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:02.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:02.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:02 smithi178 ceph-mon[103156]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T10:04:02.706 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:02 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:02.707 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/3593516681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:02.707 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:02 smithi186 ceph-mon[104393]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T10:04:02.707 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:02 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T10:04:02.787 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-08T10:04:02.787 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T10:04:02.923 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T10:04:02.966 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T10:04:02.983 INFO:tasks.cephadm:Stopping all daemons... 2023-10-08T10:04:02.983 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-08T10:04:02.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T10:04:03.451 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 systemd[1]: Stopping Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:03.451 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 ceph-mon[99491]: pgmap v570: 33 pgs: 33 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:03.451 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 ceph-mon[99491]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:03.451 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 ceph-mon[99491]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T10:04:03.702 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a[99467]: 2023-10-08T10:04:03.677+0000 7f2304d39700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:03.703 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:03 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a[99467]: 2023-10-08T10:04:03.677+0000 7f2304d39700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-08T10:04:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:03 smithi178 ceph-mon[103156]: pgmap v570: 33 pgs: 33 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:03 smithi178 ceph-mon[103156]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:03.705 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:03 smithi178 ceph-mon[103156]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T10:04:03.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:03 smithi186 ceph-mon[104393]: pgmap v570: 33 pgs: 33 active+clean; 577 KiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T10:04:03.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:03 smithi186 ceph-mon[104393]: from='client.? 172.21.15.97:0/105673727' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T10:04:03.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:03 smithi186 ceph-mon[104393]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T10:04:06.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-08T10:04:06.202 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:05 smithi097 bash[181382]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T10:04:06.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:06 smithi097 systemd[1]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-08T10:04:06.203 INFO:journalctl@ceph.mon.a.smithi097.stdout:Oct 08 10:04:06 smithi097 systemd[1]: Stopped Ceph mon.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7. 2023-10-08T10:04:06.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:06.261 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-08T10:04:06.261 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-08T10:04:06.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.b 2023-10-08T10:04:06.690 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:06 smithi178 systemd[1]: Stopping Ceph mon.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:06.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:06 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-b[103131]: 2023-10-08T10:04:06.687+0000 7f92e7ff3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:06.955 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:06 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-b[103131]: 2023-10-08T10:04:06.687+0000 7f92e7ff3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-08T10:04:07.455 INFO:journalctl@ceph.mon.b.smithi178.stdout:Oct 08 10:04:07 smithi178 bash[144708]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-b 2023-10-08T10:04:07.555 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-08T10:04:07.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:07.603 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-08T10:04:07.604 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-08T10:04:07.604 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.c 2023-10-08T10:04:08.016 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:07 smithi186 systemd[1]: Stopping Ceph mon.c for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:08.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:08 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-c[104369]: 2023-10-08T10:04:08.014+0000 7f89fb8fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:08.275 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:08 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-c[104369]: 2023-10-08T10:04:08.014+0000 7f89fb8fd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-08T10:04:08.775 INFO:journalctl@ceph.mon.c.smithi186.stdout:Oct 08 10:04:08 smithi186 bash[146955]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mon-c 2023-10-08T10:04:08.808 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-08T10:04:08.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:08.874 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-08T10:04:08.874 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-08T10:04:08.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T10:04:09.203 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 10:04:08 smithi097 systemd[1]: Stopping Ceph mgr.x for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:09.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 10:04:09 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T10:04:09.503+0000 7fc5f0080700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:09.953 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Oct 08 10:04:09 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-x[99734]: 2023-10-08T10:04:09.503+0000 7fc5f0080700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-08T10:04:11.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-08T10:04:12.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:12.014 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-08T10:04:12.014 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-08T10:04:12.014 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.y 2023-10-08T10:04:12.444 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 10:04:12 smithi178 systemd[1]: Stopping Ceph mgr.y for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:12.705 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 10:04:12 smithi178 bash[144874]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-y 2023-10-08T10:04:12.908 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Oct 08 10:04:12 smithi178 systemd[1]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-08T10:04:12.914 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-08T10:04:12.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:12.952 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-08T10:04:12.952 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-08T10:04:12.952 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.z 2023-10-08T10:04:13.275 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 10:04:12 smithi186 systemd[1]: Stopping Ceph mgr.z for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:13.661 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 10:04:13 smithi186 bash[147113]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mgr-z 2023-10-08T10:04:13.662 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Oct 08 10:04:13 smithi186 systemd[1]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-08T10:04:13.847 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-08T10:04:13.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:13.883 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-08T10:04:13.883 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-08T10:04:13.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.0 2023-10-08T10:04:14.202 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 10:04:13 smithi097 systemd[1]: Stopping Ceph osd.0 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:14.702 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 10:04:14 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T10:04:14.240+0000 7fed3bae9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:14.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 10:04:14 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T10:04:14.240+0000 7fed3bae9700 -1 osd.0 93 *** Got signal Terminated *** 2023-10-08T10:04:14.703 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 10:04:14 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0[122608]: 2023-10-08T10:04:14.240+0000 7fed3bae9700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:20.203 INFO:journalctl@ceph.osd.0.smithi097.stdout:Oct 08 10:04:20 smithi097 bash[181658]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-0 2023-10-08T10:04:21.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-08T10:04:21.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:21.618 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-08T10:04:21.618 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-08T10:04:21.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.1 2023-10-08T10:04:21.955 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 10:04:21 smithi178 systemd[1]: Stopping Ceph osd.1 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:22.455 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 10:04:21 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T10:04:21.968+0000 7f03d7507700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:22.455 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 10:04:21 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T10:04:21.968+0000 7f03d7507700 -1 osd.1 93 *** Got signal Terminated *** 2023-10-08T10:04:22.455 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 10:04:21 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1[108242]: 2023-10-08T10:04:21.968+0000 7f03d7507700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:28.206 INFO:journalctl@ceph.osd.1.smithi178.stdout:Oct 08 10:04:27 smithi178 bash[145019]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-1 2023-10-08T10:04:29.418 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-08T10:04:29.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:29.459 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-08T10:04:29.459 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-08T10:04:29.459 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.2 2023-10-08T10:04:29.775 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 10:04:29 smithi186 systemd[1]: Stopping Ceph osd.2 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:30.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 10:04:29 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T10:04:29.810+0000 7f5118576700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:30.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 10:04:29 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T10:04:29.810+0000 7f5118576700 -1 osd.2 93 *** Got signal Terminated *** 2023-10-08T10:04:30.275 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 10:04:29 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2[110223]: 2023-10-08T10:04:29.810+0000 7f5118576700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:35.775 INFO:journalctl@ceph.osd.2.smithi186.stdout:Oct 08 10:04:35 smithi186 bash[147264]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-2 2023-10-08T10:04:36.960 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-08T10:04:37.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:37.001 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-08T10:04:37.001 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-08T10:04:37.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.3 2023-10-08T10:04:37.354 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 10:04:37 smithi097 systemd[1]: Stopping Ceph osd.3 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:37.703 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 10:04:37 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T10:04:37.353+0000 7f3fc02c5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:37.703 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 10:04:37 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T10:04:37.353+0000 7f3fc02c5700 -1 osd.3 93 *** Got signal Terminated *** 2023-10-08T10:04:37.703 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 10:04:37 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3[131278]: 2023-10-08T10:04:37.353+0000 7f3fc02c5700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:43.243 INFO:journalctl@ceph.osd.3.smithi097.stdout:Oct 08 10:04:43 smithi097 bash[181928]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-3 2023-10-08T10:04:44.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-08T10:04:44.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:44.542 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-08T10:04:44.542 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-08T10:04:44.543 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.4 2023-10-08T10:04:44.907 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 10:04:44 smithi178 systemd[1]: Stopping Ceph osd.4 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:45.205 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 10:04:44 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T10:04:44.905+0000 7f09749e9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:45.205 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 10:04:44 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T10:04:44.905+0000 7f09749e9700 -1 osd.4 93 *** Got signal Terminated *** 2023-10-08T10:04:45.205 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 10:04:44 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4[116881]: 2023-10-08T10:04:44.905+0000 7f09749e9700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:51.205 INFO:journalctl@ceph.osd.4.smithi178.stdout:Oct 08 10:04:50 smithi178 bash[145287]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-4 2023-10-08T10:04:52.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-08T10:04:52.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:52.180 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-08T10:04:52.180 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-08T10:04:52.180 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.5 2023-10-08T10:04:52.525 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 10:04:52 smithi186 systemd[1]: Stopping Ceph osd.5 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:04:53.025 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 10:04:52 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T10:04:52.533+0000 7faae6e03700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:04:53.025 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 10:04:52 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T10:04:52.533+0000 7faae6e03700 -1 osd.5 93 *** Got signal Terminated *** 2023-10-08T10:04:53.025 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 10:04:52 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5[118863]: 2023-10-08T10:04:52.533+0000 7faae6e03700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:04:58.622 INFO:journalctl@ceph.osd.5.smithi186.stdout:Oct 08 10:04:58 smithi186 bash[147528]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-5 2023-10-08T10:04:59.885 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-08T10:04:59.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:04:59.925 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-08T10:04:59.925 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-08T10:04:59.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.6 2023-10-08T10:05:00.309 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 10:04:59 smithi097 systemd[1]: Stopping Ceph osd.6 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:00.702 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 10:05:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T10:05:00.308+0000 7fbe06cb4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:00.703 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 10:05:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T10:05:00.308+0000 7fbe06cb4700 -1 osd.6 93 *** Got signal Terminated *** 2023-10-08T10:05:00.703 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 10:05:00 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6[139222]: 2023-10-08T10:05:00.308+0000 7fbe06cb4700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:06.266 INFO:journalctl@ceph.osd.6.smithi097.stdout:Oct 08 10:05:06 smithi097 bash[182195]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-6 2023-10-08T10:05:07.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-08T10:05:07.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:07.567 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-08T10:05:07.567 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-08T10:05:07.567 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.7 2023-10-08T10:05:07.927 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 10:05:07 smithi178 systemd[1]: Stopping Ceph osd.7 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:08.205 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 10:05:07 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T10:05:07.925+0000 7fe429d2a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:08.205 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 10:05:07 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T10:05:07.925+0000 7fe429d2a700 -1 osd.7 93 *** Got signal Terminated *** 2023-10-08T10:05:08.205 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 10:05:07 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7[125473]: 2023-10-08T10:05:07.925+0000 7fe429d2a700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:13.955 INFO:journalctl@ceph.osd.7.smithi178.stdout:Oct 08 10:05:13 smithi178 bash[145555]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-7 2023-10-08T10:05:15.040 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-08T10:05:15.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:15.080 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-08T10:05:15.080 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-08T10:05:15.080 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.8 2023-10-08T10:05:15.430 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 10:05:15 smithi186 systemd[1]: Stopping Ceph osd.8 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:15.775 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 10:05:15 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T10:05:15.429+0000 7fddec175700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:15.775 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 10:05:15 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T10:05:15.429+0000 7fddec175700 -1 osd.8 93 *** Got signal Terminated *** 2023-10-08T10:05:15.775 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 10:05:15 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8[127492]: 2023-10-08T10:05:15.429+0000 7fddec175700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:20.953 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:20 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:20.543+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:21.215 INFO:journalctl@ceph.osd.8.smithi186.stdout:Oct 08 10:05:21 smithi186 bash[147803]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-8 2023-10-08T10:05:21.525 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:21 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:21.251+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:21.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:21 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:21.572+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:22.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:22 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:22.124+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:22.473 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:22 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:22.257+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:22.481 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-08T10:05:22.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:22.521 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-08T10:05:22.521 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-08T10:05:22.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.9 2023-10-08T10:05:22.873 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:22 smithi097 systemd[1]: Stopping Ceph osd.9 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:22.873 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:22 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:22.572+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:23.202 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:22 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:22.872+0000 7fd935423700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:23.202 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:22 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:22.872+0000 7fd935423700 -1 osd.9 93 *** Got signal Terminated *** 2023-10-08T10:05:23.203 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:22 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:22.872+0000 7fd935423700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:23.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:23 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:23.087+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:23.525 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:23 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:23.254+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:23.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:23 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:23.564+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:24.456 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:24 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:24.044+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:24.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:24 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:24.291+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:24.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:24 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:24.603+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:25.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:25 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:25.045+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:25.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:25 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:25.276+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:25.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:25 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:25.584+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:26.457 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:26 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:26.055+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:26.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:26 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:26.314+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:26.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:26 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:26.546+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:27.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:27 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:27.077+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:27.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:27 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:27.314+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:27.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:27 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9[147967]: 2023-10-08T10:05:27.508+0000 7fd930a17700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:16.398375+0000 front 2023-10-08T10:04:16.398522+0000 (oldest deadline 2023-10-08T10:05:19.898546+0000) 2023-10-08T10:05:28.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:28 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:28.083+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:28.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:28 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:28.287+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:28.952 INFO:journalctl@ceph.osd.9.smithi097.stdout:Oct 08 10:05:28 smithi097 bash[182462]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-9 2023-10-08T10:05:29.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:29 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:29.121+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:29.525 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:29 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:29.265+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:29.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-08T10:05:30.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:30.017 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-08T10:05:30.017 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-08T10:05:30.018 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.10 2023-10-08T10:05:30.381 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:30 smithi178 systemd[1]: Stopping Ceph osd.10 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:30.382 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:30 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:30.118+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:30.705 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:30 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:30.380+0000 7f40ceb7b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:30.705 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:30 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:30.380+0000 7f40ceb7b700 -1 osd.10 93 *** Got signal Terminated *** 2023-10-08T10:05:30.705 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:30 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:30.380+0000 7f40ceb7b700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:30.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:30 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:30.289+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:31.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:31 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:31.167+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:31.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:31 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:31.167+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:22.293522+0000 front 2023-10-08T10:04:22.293540+0000 (oldest deadline 2023-10-08T10:05:31.094164+0000) 2023-10-08T10:05:31.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:31 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:31.329+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:31.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:31 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:31.329+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:32.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:32 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:32.195+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:32.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:32 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:32.195+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:22.293522+0000 front 2023-10-08T10:04:22.293540+0000 (oldest deadline 2023-10-08T10:05:31.094164+0000) 2023-10-08T10:05:32.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:32 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:32.366+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:32.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:32 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:32.366+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:33.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:33 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:33.161+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:33.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:33 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:33.161+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:22.293522+0000 front 2023-10-08T10:04:22.293540+0000 (oldest deadline 2023-10-08T10:05:31.094164+0000) 2023-10-08T10:05:33.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:33 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:33.341+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:33.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:33 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:33.341+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:34.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:34 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:34.144+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:34.456 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:34 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:34.144+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:22.293522+0000 front 2023-10-08T10:04:22.293540+0000 (oldest deadline 2023-10-08T10:05:31.094164+0000) 2023-10-08T10:05:34.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:34 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:34.293+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:34.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:34 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:34.293+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:35.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:35 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:35.132+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.891908+0000 front 2023-10-08T10:04:15.891980+0000 (oldest deadline 2023-10-08T10:05:21.792854+0000) 2023-10-08T10:05:35.455 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:35 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10[133449]: 2023-10-08T10:05:35.132+0000 7f40c896c700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:22.293522+0000 front 2023-10-08T10:04:22.293540+0000 (oldest deadline 2023-10-08T10:05:31.094164+0000) 2023-10-08T10:05:35.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:35 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:35.297+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:35.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:35 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:35.297+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:36.155 INFO:journalctl@ceph.osd.10.smithi178.stdout:Oct 08 10:05:36 smithi178 bash[145831]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-10 2023-10-08T10:05:36.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:36 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:36.345+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:36.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:36 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:36.345+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:37.416 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-08T10:05:37.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:37.457 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-08T10:05:37.457 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-08T10:05:37.458 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.11 2023-10-08T10:05:37.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:37.313+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:37.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:37.313+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:37.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 systemd[1]: Stopping Ceph osd.11 for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:38.275 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:37.800+0000 7f1198fb3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:38.275 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:37.800+0000 7f1198fb3700 -1 osd.11 93 *** Got signal Terminated *** 2023-10-08T10:05:38.275 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:37 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:37.800+0000 7f1198fb3700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T10:05:38.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:38 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:38.358+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:38.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:38 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:38.358+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:39.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:39 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:39.309+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:39.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:39 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:39.309+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:39.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:39 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:39.309+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-08T10:04:33.330416+0000 front 2023-10-08T10:04:33.330342+0000 (oldest deadline 2023-10-08T10:05:39.230527+0000) 2023-10-08T10:05:40.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:40 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:40.322+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:40.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:40 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:40.322+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:40.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:40 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:40.322+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-08T10:04:33.330416+0000 front 2023-10-08T10:04:33.330342+0000 (oldest deadline 2023-10-08T10:05:39.230527+0000) 2023-10-08T10:05:41.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:41 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:41.328+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:41.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:41 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:41.328+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:41.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:41 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:41.328+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-08T10:04:33.330416+0000 front 2023-10-08T10:04:33.330342+0000 (oldest deadline 2023-10-08T10:05:39.230527+0000) 2023-10-08T10:05:42.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:42 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:42.334+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2023-10-08T10:04:15.227862+0000 front 2023-10-08T10:04:15.227901+0000 (oldest deadline 2023-10-08T10:05:21.127899+0000) 2023-10-08T10:05:42.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:42 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:42.334+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.1 since back 2023-10-08T10:04:26.429257+0000 front 2023-10-08T10:04:26.429031+0000 (oldest deadline 2023-10-08T10:05:31.128995+0000) 2023-10-08T10:05:42.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:42 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11[136252]: 2023-10-08T10:05:42.334+0000 7f11945a7700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2023-10-08T10:04:33.330416+0000 front 2023-10-08T10:04:33.330342+0000 (oldest deadline 2023-10-08T10:05:39.230527+0000) 2023-10-08T10:05:43.775 INFO:journalctl@ceph.osd.11.smithi186.stdout:Oct 08 10:05:43 smithi186 bash[148067]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-osd-11 2023-10-08T10:05:45.058 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-08T10:05:45.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:45.098 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-08T10:05:45.098 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-08T10:05:45.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.a 2023-10-08T10:05:45.453 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 10:05:45 smithi097 systemd[1]: Stopping Ceph mds.a for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:45.954 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 10:05:45 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-a[155978]: 2023-10-08T10:05:45.453+0000 7f94297b0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:45.954 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 10:05:45 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-a[155978]: 2023-10-08T10:05:45.453+0000 7f94297b0700 -1 mds.a *** got signal Terminated *** 2023-10-08T10:05:49.434 INFO:journalctl@ceph.mds.a.smithi097.stdout:Oct 08 10:05:49 smithi097 bash[182744]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-a 2023-10-08T10:05:49.440 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-08T10:05:49.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:49.478 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-08T10:05:49.479 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-08T10:05:49.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.d 2023-10-08T10:05:49.703 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 10:05:49 smithi097 systemd[1]: Stopping Ceph mds.d for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:50.202 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 10:05:49 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-d[154994]: 2023-10-08T10:05:49.851+0000 7f3a6d347700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:50.203 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 10:05:49 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-d[154994]: 2023-10-08T10:05:49.851+0000 7f3a6d347700 -1 mds.d *** got signal Terminated *** 2023-10-08T10:05:53.933 INFO:journalctl@ceph.mds.d.smithi097.stdout:Oct 08 10:05:53 smithi097 bash[182890]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-d 2023-10-08T10:05:54.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-08T10:05:54.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:54.153 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-08T10:05:54.153 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-08T10:05:54.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.g 2023-10-08T10:05:54.452 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 10:05:54 smithi097 systemd[1]: Stopping Ceph mds.g for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:54.953 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 10:05:54 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-g[155594]: 2023-10-08T10:05:54.516+0000 7f23d8ffe700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:54.953 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 10:05:54 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-g[155594]: 2023-10-08T10:05:54.516+0000 7f23d8ffe700 -1 mds.g *** got signal Terminated *** 2023-10-08T10:05:58.952 INFO:journalctl@ceph.mds.g.smithi097.stdout:Oct 08 10:05:58 smithi097 bash[183030]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-g 2023-10-08T10:05:59.021 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-08T10:05:59.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:05:59.056 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-08T10:05:59.056 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-08T10:05:59.056 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.j 2023-10-08T10:05:59.442 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 10:05:59 smithi097 systemd[1]: Stopping Ceph mds.j for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:05:59.702 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 10:05:59 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-j[154608]: 2023-10-08T10:05:59.440+0000 7fbbff836700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:05:59.703 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 10:05:59 smithi097 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-j[154608]: 2023-10-08T10:05:59.440+0000 7fbbff836700 -1 mds.j *** got signal Terminated *** 2023-10-08T10:06:03.617 INFO:journalctl@ceph.mds.j.smithi097.stdout:Oct 08 10:06:03 smithi097 bash[183180]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-j 2023-10-08T10:06:03.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-08T10:06:03.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:03.837 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-08T10:06:03.838 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-08T10:06:03.838 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.b 2023-10-08T10:06:04.200 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:06:03 smithi178 systemd[1]: Stopping Ceph mds.b for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:04.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:06:04 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: 2023-10-08T10:06:04.198+0000 7fb8ec0db700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:04.455 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:06:04 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b[140338]: 2023-10-08T10:06:04.198+0000 7fb8ec0db700 -1 mds.b *** got signal Terminated *** 2023-10-08T10:06:06.559 INFO:journalctl@ceph.mds.b.smithi178.stdout:Oct 08 10:06:06 smithi178 bash[146094]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-b 2023-10-08T10:06:06.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-08T10:06:06.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:06.783 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-08T10:06:06.783 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-08T10:06:06.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.e 2023-10-08T10:06:07.167 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:06:06 smithi178 systemd[1]: Stopping Ceph mds.e for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:07.455 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:06:07 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: 2023-10-08T10:06:07.165+0000 7f6d7cab5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:07.455 INFO:journalctl@ceph.mds.e.smithi178.stdout:Oct 08 10:06:07 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-e[139660]: 2023-10-08T10:06:07.165+0000 7f6d7cab5700 -1 mds.e *** got signal Terminated *** 2023-10-08T10:06:10.720 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-08T10:06:10.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:10.757 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-08T10:06:10.758 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-08T10:06:10.758 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.h 2023-10-08T10:06:11.141 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:06:10 smithi178 systemd[1]: Stopping Ceph mds.h for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:11.455 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:06:11 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: 2023-10-08T10:06:11.138+0000 7f85184a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:11.455 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:06:11 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h[139997]: 2023-10-08T10:06:11.139+0000 7f85184a3700 -1 mds.h *** got signal Terminated *** 2023-10-08T10:06:14.503 INFO:journalctl@ceph.mds.h.smithi178.stdout:Oct 08 10:06:14 smithi178 bash[146381]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-h 2023-10-08T10:06:14.694 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-08T10:06:14.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:14.731 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-08T10:06:14.731 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-08T10:06:14.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.k 2023-10-08T10:06:14.955 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 10:06:14 smithi178 systemd[1]: Stopping Ceph mds.k for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:15.455 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 10:06:15 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-k[139319]: 2023-10-08T10:06:15.113+0000 7fb4a383d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:15.455 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 10:06:15 smithi178 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-k[139319]: 2023-10-08T10:06:15.113+0000 7fb4a383d700 -1 mds.k *** got signal Terminated *** 2023-10-08T10:06:17.372 INFO:journalctl@ceph.mds.k.smithi178.stdout:Oct 08 10:06:17 smithi178 bash[146520]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-k 2023-10-08T10:06:17.569 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-08T10:06:17.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:17.605 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-08T10:06:17.605 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-08T10:06:17.605 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.c 2023-10-08T10:06:17.963 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:06:17 smithi186 systemd[1]: Stopping Ceph mds.c for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:18.274 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:06:17 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: 2023-10-08T10:06:17.962+0000 7f09964b1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:18.275 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:06:17 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c[142445]: 2023-10-08T10:06:17.962+0000 7f09964b1700 -1 mds.c *** got signal Terminated *** 2023-10-08T10:06:22.424 INFO:journalctl@ceph.mds.c.smithi186.stdout:Oct 08 10:06:22 smithi186 bash[148340]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-c 2023-10-08T10:06:22.431 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-08T10:06:22.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:22.469 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-08T10:06:22.469 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-08T10:06:22.470 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.f 2023-10-08T10:06:22.775 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:06:22 smithi186 systemd[1]: Stopping Ceph mds.f for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:23.275 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:06:22 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: 2023-10-08T10:06:22.840+0000 7fb0112f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:23.275 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:06:22 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f[142103]: 2023-10-08T10:06:22.840+0000 7fb0112f3700 -1 mds.f *** got signal Terminated *** 2023-10-08T10:06:26.518 INFO:journalctl@ceph.mds.f.smithi186.stdout:Oct 08 10:06:26 smithi186 bash[148499]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-f 2023-10-08T10:06:26.709 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-08T10:06:26.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:26.744 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-08T10:06:26.744 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-08T10:06:26.744 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.i 2023-10-08T10:06:27.025 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:06:26 smithi186 systemd[1]: Stopping Ceph mds.i for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:27.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:06:27 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: 2023-10-08T10:06:27.118+0000 7f8912699700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:27.525 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:06:27 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i[142786]: 2023-10-08T10:06:27.118+0000 7f8912699700 -1 mds.i *** got signal Terminated *** 2023-10-08T10:06:30.775 INFO:journalctl@ceph.mds.i.smithi186.stdout:Oct 08 10:06:30 smithi186 bash[148636]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-i 2023-10-08T10:06:30.807 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-08T10:06:30.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:30.842 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-08T10:06:30.842 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-08T10:06:30.842 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.l 2023-10-08T10:06:31.210 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 10:06:30 smithi186 systemd[1]: Stopping Ceph mds.l for 7ec07914-65bf-11ee-8db5-212e2dc638e7... 2023-10-08T10:06:31.525 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 10:06:31 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-l[141761]: 2023-10-08T10:06:31.209+0000 7f60df9dd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T10:06:31.525 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 10:06:31 smithi186 ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-l[141761]: 2023-10-08T10:06:31.209+0000 7f60df9dd700 -1 mds.l *** got signal Terminated *** 2023-10-08T10:06:35.275 INFO:journalctl@ceph.mds.l.smithi186.stdout:Oct 08 10:06:35 smithi186 bash[148783]: ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7-mds-l 2023-10-08T10:06:35.485 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7ec07914-65bf-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-08T10:06:35.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T10:06:35.521 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-08T10:06:35.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T10:06:35.828 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:06:37.468 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T10:06:37.789 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:06:39.422 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T10:06:39.727 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:06:41.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:06:41.364 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:06:41.401 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T10:06:41.432 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-08T10:06:41.434 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi097/crash 2023-10-08T10:06:41.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T10:06:41.479 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T10:06:41.479 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2023-10-08T10:06:41.481 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi178/crash 2023-10-08T10:06:41.482 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T10:06:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T10:06:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2023-10-08T10:06:41.538 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi186/crash 2023-10-08T10:06:41.539 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T10:06:41.584 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T10:06:41.584 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2023-10-08T10:06:41.585 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-08T10:06:41.586 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-08T10:06:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:grep: /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-08T10:06:41.623 INFO:tasks.cephadm:Compressing logs... 2023-10-08T10:06:41.623 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T10:06:41.666 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T10:06:41.668 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T10:06:41.702 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-08T10:06:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-08T10:06:41.705 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-08T10:06:41.705 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-08T10:06:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-08T10:06:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.3.log: /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-08T10:06:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.9.log: /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-08T10:06:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.log: 94.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-08T10:06:41.713 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-08T10:06:41.714 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-08T10:06:41.716 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-08T10:06:41.716 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-08T10:06:41.717 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T10:06:41.718 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-10-08T10:06:41.718 INFO:teuthology.orchestra.run.smithi178.stderr:-5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-08T10:06:41.719 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-08T10:06:41.719 INFO:teuthology.orchestra.run.smithi186.stderr: 94.6% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-08T10:06:41.723 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-08T10:06:41.723 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-08T10:06:41.724 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-08T10:06:41.724 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-08T10:06:41.724 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-08T10:06:41.724 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T10:06:41.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-08T10:06:41.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-08T10:06:41.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-08T10:06:41.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-08T10:06:41.726 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T10:06:41.727 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-08T10:06:41.730 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-08T10:06:41.759 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T10:06:41.762 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-08T10:06:41.764 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T10:06:41.860 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T10:06:42.118 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T10:06:42.212 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-08T10:06:42.268 INFO:teuthology.orchestra.run.smithi186.stderr: 94.7% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-08T10:06:42.453 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T10:06:42.693 INFO:teuthology.orchestra.run.smithi097.stderr: 89.9% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-08T10:06:42.698 INFO:teuthology.orchestra.run.smithi178.stderr: 91.0% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T10:06:42.888 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-08T10:06:43.323 INFO:teuthology.orchestra.run.smithi186.stderr: 91.8% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-08T10:06:43.558 INFO:teuthology.orchestra.run.smithi178.stderr: 94.7% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-08T10:06:44.042 INFO:teuthology.orchestra.run.smithi097.stderr: 90.6% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-08T10:06:48.939 INFO:teuthology.orchestra.run.smithi178.stderr: 93.4% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-08T10:07:10.568 INFO:teuthology.orchestra.run.smithi186.stderr: 97.4% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-08T10:07:54.246 INFO:teuthology.orchestra.run.smithi186.stderr: 98.1% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-08T10:10:07.172 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-08T10:10:13.524 INFO:teuthology.orchestra.run.smithi178.stderr: 92.4% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-08T10:10:24.758 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-08T10:10:47.438 INFO:teuthology.orchestra.run.smithi178.stderr: 92.4% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-08T10:11:03.588 INFO:teuthology.orchestra.run.smithi186.stderr: 92.1% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-08T10:11:33.876 INFO:teuthology.orchestra.run.smithi178.stderr: 92.4% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-08T10:12:07.128 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-08T10:12:11.288 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-08T10:12:11.290 INFO:teuthology.orchestra.run.smithi178.stderr: 2023-10-08T10:12:11.290 INFO:teuthology.orchestra.run.smithi178.stderr:real 5m29.606s 2023-10-08T10:12:11.290 INFO:teuthology.orchestra.run.smithi178.stderr:user 7m35.092s 2023-10-08T10:12:11.290 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m17.036s 2023-10-08T10:12:25.405 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-08T10:12:25.613 INFO:teuthology.orchestra.run.smithi186.stderr: 91.7% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-08T10:12:25.645 INFO:teuthology.orchestra.run.smithi186.stderr: 2023-10-08T10:12:25.645 INFO:teuthology.orchestra.run.smithi186.stderr:real 5m43.961s 2023-10-08T10:12:25.645 INFO:teuthology.orchestra.run.smithi186.stderr:user 7m46.734s 2023-10-08T10:12:25.645 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m18.220s 2023-10-08T10:12:28.349 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-08T10:13:17.427 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/7ec07914-65bf-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-08T10:13:17.429 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-10-08T10:13:17.430 INFO:teuthology.orchestra.run.smithi097.stderr:real 6m35.750s 2023-10-08T10:13:17.430 INFO:teuthology.orchestra.run.smithi097.stderr:user 8m53.294s 2023-10-08T10:13:17.430 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m19.972s 2023-10-08T10:13:17.430 INFO:tasks.cephadm:Archiving logs... 2023-10-08T10:13:17.431 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi097/log 2023-10-08T10:13:17.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T10:14:46.342 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi178/log 2023-10-08T10:14:46.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T10:16:01.734 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi186/log 2023-10-08T10:16:01.735 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T10:17:18.365 INFO:tasks.cephadm:Removing cluster... 2023-10-08T10:17:18.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force 2023-10-08T10:17:19.020 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:17:20.057 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force 2023-10-08T10:17:20.857 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:17:21.840 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7ec07914-65bf-11ee-8db5-212e2dc638e7 --force 2023-10-08T10:17:22.404 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 7ec07914-65bf-11ee-8db5-212e2dc638e7 2023-10-08T10:17:23.437 INFO:tasks.cephadm:Removing cephadm ... 2023-10-08T10:17:23.438 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T10:17:23.458 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T10:17:23.496 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T10:17:23.522 INFO:tasks.cephadm:Teardown complete 2023-10-08T10:17:23.522 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T10:17:23.533 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T10:17:23.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T10:17:23.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T10:17:23.539 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T10:17:23.675 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T10:17:23.685 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T10:17:23.686 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T10:17:23.688 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T10:17:23.691 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T10:17:23.705 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-10-08T10:17:23.706 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-08T10:17:23.707 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-10-08T10:17:23.736 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T10:17:23.736 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-08T10:17:23.736 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 -502us[ -502us] +/- 88ms 2023-10-08T10:17:23.736 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 +293us[ +328us] +/- 45ms 2023-10-08T10:17:23.737 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +13us[ +48us] +/- 72ms 2023-10-08T10:17:23.737 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 -500us[ -494us] +/- 88ms 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 +314us[ +320us] +/- 46ms 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +12us[ +18us] +/- 72ms 2023-10-08T10:17:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T10:17:23.742 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T10:17:23.742 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-10-08T10:17:23.742 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 -485us[ -485us] +/- 88ms 2023-10-08T10:17:23.743 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 +322us[ +347us] +/- 45ms 2023-10-08T10:17:23.743 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +9340ns[ +34us] +/- 72ms 2023-10-08T10:17:23.743 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T10:17:23.744 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T10:17:23.754 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T10:17:23.755 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T10:17:23.769 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T10:17:23.996 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T10:17:24.244 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T10:17:24.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T10:17:24.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:24.451 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T10:17:24.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:24.486 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T10:17:24.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:24.518 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T10:17:24.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T10:17:24.541 INFO:teuthology.task.internal:Duration was 2596.247143 seconds 2023-10-08T10:17:24.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T10:17:24.552 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T10:17:24.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T10:17:24.555 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T10:17:24.557 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T10:17:24.624 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T10:17:24.624 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T10:17:24.625 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T10:17:25.043 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T10:17:25.043 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-10-08T10:17:25.043 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T10:17:25.070 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2023-10-08T10:17:25.070 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T10:17:25.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-10-08T10:17:25.111 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T10:17:25.142 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T10:17:25.142 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T10:17:25.145 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T10:17:25.153 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T10:17:25.254 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T10:17:25.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T10:17:25.257 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T10:17:25.261 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T10:17:25.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T10:17:25.403 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T10:17:25.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T10:17:25.438 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T10:17:25.470 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T10:17:25.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T10:17:25.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.513 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.515 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.546 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-10-08T10:17:25.552 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2023-10-08T10:17:25.586 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-10-08T10:17:25.600 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:25.624 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:25.643 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T10:17:25.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T10:17:25.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T10:17:25.671 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T10:17:25.672 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi097 2023-10-08T10:17:25.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T10:17:25.715 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi178 2023-10-08T10:17:25.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T10:17:25.753 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807/remote/smithi186 2023-10-08T10:17:25.753 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T10:17:25.792 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T10:17:25.792 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T10:17:25.794 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T10:17:25.797 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T10:17:25.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T10:17:25.855 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T10:17:25.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T10:17:25.874 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T10:17:25.874 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T10:17:25.877 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T10:17:25.880 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T10:17:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 10:17 /home/ubuntu/cephtest 2023-10-08T10:17:25.893 INFO:teuthology.orchestra.run.smithi178.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 10:17 /home/ubuntu/cephtest 2023-10-08T10:17:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 10:17 /home/ubuntu/cephtest 2023-10-08T10:17:25.899 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T10:17:25.913 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T10:17:25.941 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T10:17:25.969 DEBUG:teuthology.nuke:shortname: smithi097 2023-10-08T10:17:25.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T10:17:25.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.795994', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T10:17:25.996 DEBUG:teuthology.nuke:shortname: smithi178 2023-10-08T10:17:25.997 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T10:17:26.025 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.798030', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T10:17:26.027 DEBUG:teuthology.nuke:shortname: smithi186 2023-10-08T10:17:26.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T10:17:26.052 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417807', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 09:28:13.799528', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T10:17:26.064 INFO:teuthology.orchestra.console.smithi097:Power off 2023-10-08T10:17:26.064 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T10:17:26.091 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2023-10-08T10:17:26.091 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:26.107 INFO:teuthology.orchestra.console.smithi178:Power off 2023-10-08T10:17:26.107 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T10:17:26.118 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-10-08T10:17:26.128 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-10-08T10:17:26.128 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:26.146 INFO:teuthology.orchestra.console.smithi186:Power off 2023-10-08T10:17:26.146 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T10:17:26.154 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-10-08T10:17:26.167 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-10-08T10:17:26.167 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:26.190 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-08T10:17:30.119 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:30.156 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:30.191 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:30.232 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-10-08T10:17:30.266 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-10-08T10:17:30.301 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-08T10:17:34.234 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:34.266 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:34.302 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:34.344 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-10-08T10:17:34.378 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-10-08T10:17:34.413 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-08T10:17:34.445 INFO:teuthology.orchestra.console.smithi097:Power off completed 2023-10-08T10:17:34.478 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-10-08T10:17:38.418 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T10:17:38.528 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-10-08T10:17:38.628 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-10-08T10:17:38.733 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} duration: 2596.2471430301666 failure_reason: 'error during scrub thrashing: Command failed on smithi097 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status''' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-10-08T10:17:38.734 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T10:17:38.819 INFO:teuthology.run:FAIL